Repeatedly calling IMediaSeeking::SetPositions to framestep backwards one frame at a time will give you reverse playback. It will be very unlikely to be smooth, but I think it's going to be the best you can achieve without writing multiple custom filters.
Using a codec without any temporal compression like mjpeg or jpeg2000 would help a lot with this technique. For regular modern codecs lowering the resolution and bitrate would help.
I'm not sure that the stock AVI demux will support negative rates for any format, even if the format does not use temporal compression. And certainly most other demux filters will not support negative rate. So you will probably need a custom demux filter to do this. If you do this, make sure you use a container format with an index (mp4 for example).
If you use a format without temporal compression (motion jpeg, i-frames only mpeg-2 etc), you will pay a considerable price in frame compression in order to be able to play in reverse. If you want to do this really well, consider the approach taken by some DVD implementations. Some of them play in reverse by playing only the i-frame, so you have 1 or 2 fps. But the top-end implementation will decode a whole GOP (approx 15 frames) and will then render these frames in reverse order while decoding the previous GOP. This is expensive in development terms and in system resources, but gets a good end result.
(Adapted from
http://stackoverflow.com/q/1725534/1758762[
^])