This is another way to go that's perfectly fine. But when someone mentions a Pi, I skew towards pure value/$ and let them correct me if they want more. You'll see what I mean.
I know you said you wanted to keep your options open. If you think you might want to turn this into a gaming PC, these boxes WILL NOT work for that for various reasons. The idea behind them is that they're so cheap that it becomes absurd. However, they're extremely popular for turning into servers because they're cheap, reliable, and in Small Form Factors. The upgrades you would want for something like a media server are easily done.
Some people like the HPs. I do not like their products in general. If you want to look at those, they call them Mini Business Desktops. When an older member of my family wants a computer, I personally grab a Dell or Lenovo box like this. Both have a rep for ease of compatability with Linux.
I'll assume you can figure out which processor you might want given that you want to learn Linux. Sometimes these SFF boxes only come with DisplayPorts. No HDMI. You can of course fix this easily externally, but it costs extra money and you might have been better off just getting a more expensive model. Also pay extremely close attention to the storage. Many of these are dirt cheap because they only have an HDD. You can, however, add a dirt cheap SSD. The sales are absurd. Figure you can find a 500 GB SSD for $30 and a TB well under 100.
Before you or I actually dig into sellers, what I need to know is whether I'm on the right track or not here. Here are some examples:
SSD
Max Turbo Speed 3.8 GHz. Model 7050. Product Line Dell OptiPlex. Manufacturer Dell, Inc. Type Core i5. Clock Speed 3.4 GHz. HARD DRIVE. Installed Size 16 GB.
www.ebay.com
HDD
Find many great new & used options and get the best deals for Dell OptiPlex 7050 SFF Core i5 7500 3.4 GHz 8GB RAM 500GB HD Win 10 Pro Wifi at the best online prices at eBay! Free shipping for many products!
www.ebay.com
More RAM and bigger SSD
1TB Solid State Drive. 32GB DDR3 RAM. Our techs will be happy to assist and troubleshoot hardware errors over the phone. We can custom build the units to suit your specific needs. We carry a wide variety of options in our 45,000 sq ft facility.
www.ebay.com