Repurposing. Yes, we think that converting a book into an app is "repurposing." Repurposing, as the word suggests, refers to re-using the work for another purpose--for example, posting it at a website. Converting a book into an app would qualify, as well. There are many outstanding issues though. One is the relationship between your letter and your agreement. Some agreements are considered to be fully "integrated." That means the parties to a contract intend that their agreement to be the complete and final expression of their deal. In that case, with a few exceptions, no contradictory or supplementary pre-contract discussions or documents can be introduced. Post contract discussions, documents, or issues will be considered unless the agreement has a provision specifying how amendments are to be made and the post-contract modification doesn't meet the requirements -- provisions that are not always enforced. (Whew... that's confusing, huh? See why we think you might benefit from having someone look at the paperwork?) We assume you were supposed to receive a one-time payment for your work, not a continuing royalty for books sold. If it's the latter, then you may have additional issues as to how you're supposed to be paid for the app.
Copyright in the designs. We're also concerned about your claim to copyright ownership. We hope that the agreement you signed spells out your ownership claim. We hope it does not say things like you assign all rights, or that your contribution is a work made for hire. A lawyer might be concerned about other things as well -- for example, did the publisher disclaim your work, when registering the copyright? (You can research the copyright registration here.) On the way other hand, it's possible that some of your contributions (for example, the wording explaining how to do a stitch) may not be protected under copyright principles as copyright does not protect facts, especially when there is a limited number of ways of expressing them.