A common solution is to use a chain tensioner:
These normally screw into the rear mech boss and use a sprung lever to tension the chain, usually by an jockey wheel or similar, running against the chain. It's even possible to use an old rear mech as a chain tensioner (e.g. by running the chain round the lower jockey wheel only).
However, I don't like these ideas much, as the silent, simple, smooth, single speed feel can't be fully achieved when there's a wheel tensioning the chain.
One option is to forget about single speed, and use a cassette and rear derailleur. It so happens that I have some wheels that are nutted solid axle, the rear having a screw-on freewheel, and I have a suitable 6 speed block. The only thing that bothers me is the possibility that at extreme angles, the chain may drop off the chainwheel. A prevention mechanism for that is to use a chainset with inner and outer chain guards, but that may be hard to find at a reasonable price. Another option there is to fit an old front mech to act as a chainwheel chain keeper.
The concept for this build is to keep it simple, cheap, and that means re-using parts that I already have, so far as possible. It looks like I'm going to have to experiment quite a bit. As I said, this may end up a weird bike!