Climatically speaking, my least favorite parts of the year are when it's slightly too cold for short-sleeved shirts and slightly too warm for sweaters and jackets. In those few weeks, I check weather forecasts each morning to see the day's high and low temperatures. My general rule of thumb is to wear a sweater if the temperature's not going to rise above 70°F (21°C).
50°F (10°C) means I'll probably be cold even if I do have my sweater. Now that I'm living in a more northerly part of the world, it seems that I'll have to make some more rules of thumb about what happens below 50°F.
There's also windchill, but I just take the luck of the draw for that.