Copyright issues. Under Section 113(c) of the Copyright Act, you wouldn't need permission to reproduce the costumes (and let's hope the Dear Rich staff doesn't need it either for our 1941 blog imagery). And we also think you have a strong fair useargument for using the Captain American costume imagery as well.
What about the other stuff? The use of the image on your cover could possibly trigger trademark issues. If Marvel can argue that consumers are likely to be confused -- that is, they are likely to think Marvel had something to do with your work, Marvel could take you to court to stop the publication. We haven't seen the imagery you are using but we think the statistical chances of getting hassled by Marvel are low (assuming Marvel's lawyers read poetry). However, you should definitely get permission from the photographer for reproducing his image. And it would be wise, because you will be using the cover to advertise and sell your book, that you obtain a release from the Dad and son in the photo.