While people are working later in life, people are not working later in life at the same jobs or even in the same careers that they started in unless they are professionals like lawyers or doctors.
Corporate America is always having higher paid mid level management types taking early retirement so they can hire college grads at 1/3 to 1/2 the cost. Since these types have also had their retirements obliterated by the stock market in recent years, they also have to work, so they are grabbing much lesser paying jobs or part time jobs to offset the decrease in income level.
Corporate America, simply put, doesn't allow employees to work for them for 40-50 years anymore or deep into their 60's or early 70's. The get rid of them with early retirements and talks of downsizing and then hire younger workers. Unless you are a CEO, COO, or major officer at a corporation, you aren't going to be growing old there. Not in today's employment market.
So, I do believe that a good portion of the people who stop looking for work are older, close to retirement age people. Yes, you here a lot about the people who stop looking and become homeless and so forth, but I believe they are in the minority as compared to the older workers that decide that the world is telling them its time to retire.
BTW, this exact thing has happened to 4 people I know including my father and that's why they left the workforce. I know no one who has been unemployed for such a long period of time that they gave up trying to find work.