What's the closest hotel in walking distance to the museum that's not too pricey

There really isn't a hotel in DC that isn't pricey. The I would venture to guess that the Holiday Inn a couple of blocks away would be the least expensive. Though you really only need a hotel close to a metro station, as the museum is located a block away from a blue/orange/sliver line station (Federal Center SW).

Just try google to find out.

Just do a hotel near the train station

Hyatt