Dispose those objects!
If you're using the SharePoint object model anywhere, make sure that you dispose any objects that you create (not those that the SharePoint object model creates for you, like calling GetContextWeb()).
These objects have a small memory footprint visible to the Garbage Collector, but a large amount of unmanaged resources (effectively not visible to the Garbage Collector).
If you don't dispose of them, the GC will not be in any rush to free up the memory used by these objects since it can't (or won't) see the bigger picture. The result? OutofMemoryExceptions when doing any operation that creates a lot of these objects and not disposing them.
(Side track : this is what happens to the SqlCeCommand objects).
See the full details on "Best Coding Practices" for SharePoint in this Microsoft article.
One thing's for certain - we can't ignore the unmanaged world yet...