Many give up because they get fed up with arriving at an event and learning for the first time that they are five seconds off what they thought was a good time!
I'm not convinced that folk "give up".
It's obvious to me that if you can afford the time to practise you will improve. Unfortunately a lot of folk simply don't have the time.
I second that, what with gardening and preparing "estate" for winter + probems etc
, I'm finding it difficult to practice, particularly with an understeering car and tight corner cutting rules.
My timed laps have been minimal. Not withstanding that I'll have a bash tonight but don't feel hopeful.