(1) Don't use the other company's name for purses or other fashion accessories. We assume when you say that the name is "not trademarked" (more on the improper use of trademark as a verb, later), that you mean it is not federally registered. A name can function as a trademark whether or not it is registered. In other words, if someone is using a name as a trademark, that person can stop later users even without a registration. (It's sometimes referred to as a common-law trademark.) You get additional rights and benefits by federal registration, but, as a general rule, you could not seek registration if you are aware that a competitor has been using the same name prior to your use. That's because the applicant must sign an oath indicating that the applicant has the right to claim the mark -- a right that you apparently don't have.
(2) Copyright has nothing to do with it. You can't copyright a name or short phrase. So neither you nor your competitor can protect the name mark under copyright law.
(3) What happens if you don't pay any attention to the Dear Rich blog? If you proceed with the use of the name while another company is using it, you have placed a serious limitation on your purse business. It's always possible that the other company may not find out about your use but if they do (which is more likely, the more popular you become), then you will probably have to stop your use and perhaps destroy any merchandise that contains the mark.
(4) Maybe we're wrong. We don't have all the facts, here, so it's possible we're wrong. For example, the namemay not be capable of functioning as a trademark for some reason in which case both companies can use it without much problem although neither company can stop others.