The biggest driver is the "demographic cliff" where the number of college age folks is declining.
You're correct that the problem does not affect all colleges/universities equally, as the top schools continue to be fine and less expensive colleges (and online schools) tend to still attract students due to comparatively lower costs. The ones who are suffering the most are smaller schools, particularly private ones, which can't really cut/contain costs and don't have a real way to attract sufficient students in a period of falling enrollments.