DeeKay, there are editors and there could be something like you describe as graphics compilers.
It's a bit much to ask for some program that calculates the best possible combination of sprites in whatever mode and in whatever intelligently arrange pattern to have your pixels arranged.
I think to make this a little bit feasable, you'd have to settle with something that lets you arrange sprites by hand. The rest would have to be real time graphician's brain power.
Don't scare off whatever coder would have the guts to even start this project ;-)