Uh, my info wasn't helpful before
???
The door speaker is actually mounted in a molded plastic "basket" bolted to the door sheet metal. The speaker is not mounted to the trim. The speaker grill on the trim panel really just "covers" the speaker.
BTW, the grill portion itself simply has some tabs that are inserted into slots on the trim. It is metal and the tabs are simply bent over to retain them to the door panel. Would be very easy to remove with the door panel off if you ever wanted to.
Founding member of the "No Hummer Krew"