#17Clean up manager, so we don't use nodes for everything

Some of the stuff in there does not need to be a node, instead maybe use RefCounted?