derivatives, for example) with "transformative uses" of those works. We find the use of the term "transformative" to be fairly slippery, but it generally applies to the purpose and character of the use -- for example, are you using the work to make a statement that conflicts with the original intention of the artist? (Here's an article we wrote summarizing some transformative/fair use cases.)
Fair use. Also the fact that the work is not for sale probably won't make much difference in a fair use analysis -- it is still an unauthorized display and you are benefiting from the exhibit as an artist, either financially or professionally. Don't get us wrong; we're on your side and would like to encourage your creative uses. But we haven't seen your work and know that in the end -- even if you have a reasonable fair use claim, it won't make much difference if the copyright owners want to prevent your use.
Who's on the hook? Chances are fair to good that the copyright owners -- probably the publishers -- won't know about or care about your use. Unless the use is considered offensive or of depriving them of income, they probably won't be motivated to speed dial general counsel. But if they do care, your fair use arguments will be buried under the costs of making your arguments in a court room. Alas, it is in front of a judge that fair use arguments are decided. In addition, check your agreement with the museum. Finally, you may have second thoughts about the project if you are required to indemnify the exhibitor for any third party claims of infringement.
BTW - first use/ first sale -- You refer to "first use" when describing the books that you purchased. We assume you mean "first sale," a copyright doctrine that lets you sell or dispose of authorized copies that you purchased. The first sale doctrine does not permit copying, however, of the cover or contents of the books.