I would say the most important is buying as much speaker as you can afford but it has to be an appropriate speaker for your listening space. Not too big. So many times you see people plunking tons of money on speakers that are really meant to be used in much bigger acoustical spaces. Not good.
A pair of smaller monitors that can be augmented with a sub is a much better way to go because you have control over the bottom octaves and not have to deal with muddy, boomy, unruly bass that ruins the whole experience when your speaker just doesn't like the constrains of your room. As far as supporting gear, if you can't afford a nice amp today, buy one that is 20 years old, the amps from 20 years ago will still sound great today if it was a good amp to begin with. Same with preamps.
Digital would be the exception but there are plenty of DACs and CDPs that are still very musical today if not the latest statement in resolution and transparency. Key is to find a
musical sounding DAC you can buy for next to nothing. As they say, "Rome wasn't built overnight". Piece it together as the budget becomes available.