You can drive down to Mansfield via Whitfield and Tolmie and stay around Jamieson,there are a few free sites along the creek to Warburton via Woods Point. It is a magnificent trip, the crystal clear mountain streams sparkle.
I leave the caravan behind and sleep overnight with a mattress in the back of the troopy and save myself the stress of towing a van over steep narrow mountain roads, not to mention saving myself the stress of having to fork out for the cost of a new gear box.
After Gaffney's creek the road is narrow and very windy. It's not too corrugated until after Woods Point. Coming out of Woods Point it climbs up to Matlock. The road isn't great. Once through there it's a windy road through Jericho. Once you get through that you've got a great run across the ridge of the great divide from 'the Camp' where those who built the Thompson Dam lived. It can get badly rutted on the corners...and there are many.
Once your at Cumberland Junction it's a downhill bitumen drive to the Warburton valley and then into Melbourne.
From just south of Yarck, you could go to Alexandra, then Taggerty - Healesville - across to Maroondah Highway - but that route takes you up and over the Black Spur - steep and bendy.
I drive this country several times a year and enjoy the trip every time. I plan to tour this area next week.