You're not showing the entire ball of code, but what I think you're doing is creating a context instance when your app starts and then never destroying it until your app exists.
DO NOT DO THIS. Create a context instance when you need to query the data and then dispose the context when your done with in.
EF caches everything it sees from the database. If you request an entire table, the entity tracker loads every record in the table and rehydrates it in memory. All future requests for that data come from the tracker, not the database.
Your code should look more like this:
Dim result As IEnumerable(Of Product)
Using (context As MyDbContext = new MyDbContext())
result = context.Products.ToList()
End Using