We took our 1 yr old and 3 year old to a Hummer Happening last month or so. Neither of them had any problem. Of course, with the Hummer Happening, you're in more of a controlled environment. Our vehicle didn't jolt any more than what you'd get going over a speed bump... The rocking motion even put them both to sleep... and their heads hung a little, but realistically...their heads weren't heavy enough to damage anything. My kids get more injuries from playing in the yard than off-roading.
Of course, if you've got a kid with a HUGE head, then that might be an issue...
I do agree though that the parents are the ones that should decide.