To display the heatmap, the application must retrieve a large set of records from Google servers. By browsing your history, you already have local records (from to ).
You can do a fresh start by deleting all records stored locally (this will only delete records cached locally in your browser). This is useful if you manage your Latitude history though an other application.
Once you think you have enough data, you can draw the heatmap. For now heatmap is based on location records number and not on the time spent on each location.