The function you are looking for is CDocument:SetPathName(LPCTSTR lpszPathName, BOOL bAddToMRU)
This is the function responsible for adding files to the MRU. If I remember correctly it is virtual so you can provide your own implementation in your document class's implementation.
This can check the file extension and other parameters to see whether the call to
AfxGetApp()->AddToRecentFileList(m_strPathName);
should be performed or not.
Depending on the version of VC you are using, you should base the start code of your function as a copy of the original CDocument implemenattion which you can then modify in your derived CDocument class.