You need to get an alignment.
Mine worked the same way, until I took my truck in to see the Transfer Case Whisperer. Now, it unlocks immediately.
Just kidding of course, mine does work just as you described, as did my Tahoe (which was a real PITA to get out of 4-Low).