No matter what time of day, you're going to hit traffic. The 401 is a disaster, especially though the stretch between the 400 and DVP where you would be driving. You could take the 407 over to the 404 (which turns into the DVP), but I wouldn't advise it, because the tolls are insane (and yes, even as an American, you'll either get the bill in the mail or be hit up when you go new plates, because of a mutual agreement between Ontario and NY state). And by insane, I mean more than $10 to go 20km (the government sold off the road for pennies on a dollar to some private swindlers in the late 1990's). You're going to hit traffic on the DVP too, particularly around Eglinton. Once you get to the Gardiner, you'll be fine. Word of Caution: Learn
the rules around streetcars. It's an extra challenge that most visitors aren't used to dealing with. Treat it almost like a school-bus. Always stop behind the open doors.
There is also a commuter train from Barrie (and stops south of it) to Downtown, but they only run a few trains per day, so it's more geared to the business crowd (work in the morning/home at night) and not ideal for your situation. Once you do make it Downtown, I suggest find a place to park and walking/using public transit to get around. If you try to dot around from one place to another, you'll have to find (& pay for) parking every where you go. The TTC offers a
day pass and a weekly pass. The Yonge Subway is good for N/S travel but in Downtown, E/W you're probably going to have to either hail a cab, use a streetcar, or walk.
In terms of things to do, June-September is peak festival season, so there will be no shortage of events ongoing.
Buskerfest is taking place around when you're in town, as well as the
Canadian National Exhibition. You're sadly coming a couple weeks late for the Greek Festival (one of Toronto's best) and a couple early for the Polish & Ukrainian ones. You'll want to see the usual attractions, the CN Tower, the HHOF, apparently Ripley's Aquarium is fun though I've never been, Centre Island is nice. Some others:
St. Lawrence Market, the
Distillery District is a hidden gem albeit on the far eastern edge of Downtown (how times have changed, we used to tell people not to go east of Yonge),
Fort York, the
Royal Ontario Museum,
Art Gallery of Ontario if you're an artsy individual, and if you're looking for a unique museum experience (in North America, at least), the new
Aga Khan Museum holds one of the most exclusive collections of Islamic arts and antiquities outside of the Middle East, although it's a bit of a trek (the Ontario Science Centre is also across the street, it's nothing special but is fun for kids). You could also check out a Blue Jays game, and while the Skydome isn't a "dump", it is the last multi-purpose buildings in the MLB. As a Mets fan, it might make you feel slightly nostalgic for Shea Stadium... for a little while at least.
There's no shortage of things to see and especially as you widen beyond Downtown. The city's mix of ethnic neighborhoods is probably only rivaled by New York and if you're willing to make a bit of a trek, eating dinner in Greekdown (Pape or Donlands subway stops) or in Little Italy on College St. is a nice way to spend an evening. Parks like the Scarborough Bluffs or the Niagara Peninsula in Hamilton or obviously Niagara Falls are great excursions (the Canadian side > the American side, not only for the view, but Niagara Falls itself is pretty vibrant, with kind of a Vegasy/Atlantic City feel). If you're a wine drinker, the Niagara region also offers some great wine tours. Honestly, if you're only going to be here for a short time, you might just consider taking one of those tourist hop on/hop off buses. That way you get to see the whole Downtown and get transportation to the main attractions.
Toronto isn't New York -- no city in North America is -- but it's got it's own vibe. I hope you enjoy your trip and if you do go on to Montréal (and haven't yet been), I think you'll really enjoy it as well. It's a great city, in spite of the taint the Habs bring to it.