Re: IE automation question - .NET (Core and Framework) Discussion Boards

Pete,

I appreciate the fact that you have taken the time to respond to my question. If you take a brief moment to read the post you responded to you would note that I am trying to download files from the website of Federal Reserve Bank of St. Louis. While they are a great bunch of folks, I highly doubt that they would put up an ftp server for no other reason than to resolve my little issue Laugh | :laugh:

.

If you feel the code is useful, please post a specific example of how it might be used in the context I described.

I am open to resolutions of any kind. I have done a fairly extensive search and no one seems to have a solution.

Is there any particular reason why you wish to automate IE to achieve this instead of the HTTP classes from the base class library?

--
gleat
http://blogorama.nerdworks.in[^]
--

Thanks for your suggestion - I will use any method that works. Simpler = better.
I found some examples of using a stream object - I do not understand how to:
a) convert the stream to a mshtml document
b) post the document back to the server
c) handle the subsequent file download.

How does the server return the Excel document in response to the POST? Does it send it back as a file download, i.e., does the browser show up the file download dialog? Or does it show a link that you can click?

--
gleat
http://blogorama.nerdworks.in[^]
--

The browser shows the download dialog (save file as...)

Here's a sample program that does what you describe. It does a POST to a web page and then gets a file in response which it saves to the local disk. Hope this helps!

using System;
using System.IO;
using System.Net;
using System.Collections.Generic;
using System.Text;

namespace PostDownload
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                string url = "http://nerdworks.in/webstuff/getfui.aspx";
                string postdata = "fn=logo.gif&url=http%3A%2F%2Fwww.google.com%2Fintl%2Fen_ALL%2Fimages%2Flogo.gif";

                //
                // create a web request for our page
                //
                WebRequest request = WebRequest.Create(url);
                request.ContentType = "application/x-www-form-urlencoded";
                request.Method = "POST";
                ((HttpWebRequest)request).Referer = url;

                //
                // write the post data
                //
                using (Stream stream = request.GetRequestStream())
                {
                    stream.Write(Encoding.UTF8.GetBytes(postdata), 0, Encoding.UTF8.GetByteCount(postdata));
                    stream.Flush();
                }

                //
                // get the response
                //
                WebResponse response = request.GetResponse();

                using (FileStream dest = new FileStream(GetFilename(response), FileMode.Create))
                {
                    using (Stream src = response.GetResponseStream())
                        CopyStream(src, dest);
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("ERROR: {0}", ex.ToString());
            }
        }

        private static string GetFilename(WebResponse response)
        {
            //
            // try and get a file name from Content-Disposition
            //
            string filename = response.Headers["Content-Disposition"];
            if (filename != null)
            {
                string[] tokens = filename.Split(';');
                if (tokens.Length == 2)
                {
                    tokens = tokens[1].Split('=');
                    if (tokens.Length == 2)
                        filename = tokens[1];
                    else
                        filename = null;
                }
                else
                    filename = null;
            }

            //
            // if that didn't work; try to get it from Content-Type
            //
            if (filename == null)
            {
                filename = response.Headers["Content-Type"];
                if (filename != null)
                {
                    string[] tokens = filename.Split(';');
                    if (tokens.Length == 2)
                    {
                        tokens = tokens[1].Split('=');
                        if (tokens.Length == 2)
                            filename = tokens[1];
                        else
                            filename = null;
                    }
                    else
                        filename = null;
                }
            }

            //
            // if there's still no file name then make one up
            //
            if (filename == null)
                filename = "ding.dat";

            return filename;
        }

        private static void CopyStream(Stream src, Stream dest)
        {
            byte[] buffer = new byte[1024];
            int read = -1;

            while (read != 0)
            {
                read = src.Read(buffer, 0, buffer.Length);
                if (read != 0)
                    dest.Write(buffer, 0, read);
            }
        }
    }
}

--
gleat
http://blogorama.nerdworks.in[^]
--

gleat you are amazing! Thank you very much! I am going to try to make this work. I am sure I will have some questions but I am going to work with this a while first.

Gleat, thanks again for your program. The only thing that is different in the situation I described is that the postdata values are not known when the program is run. You gave me the info I needed to look that up so all is good.

For the benefit of any one else who is trying to do this:

Download a program called Fiddler from www.fiddler2.com.

Open fiddler then use your web browser to click the elements on the page you are trying to automate. Observe the data posted back to the server. The ParseDocument function you write (see code below) will need to create a string that matches the postdata.

Here is the code you can run to automate your page. Create a form with a textbox and a button. Paste the code below as the form class. The ParseDocument function is specific for my purpose. I included it to demonstrate the basic foreach loop and id test.

<code>
public partial class Form3 : Form
    {
        public Form3()
        {
            InitializeComponent();
        }

        private void Form3_Load(object sender, EventArgs e)
        {

        }

        private void button1_Click(object sender, EventArgs e)
        {
            try
            {
                // Create a web request for the initital query to get post values
                string url = textBox1.Text;
                IHTMLDocument2 doc = new HTMLDocumentClass();
                WebRequest request1 = WebRequest.Create(url);
                WebResponse response1 = request1.GetResponse();
                StreamReader reader = new StreamReader(response1.GetResponseStream());
                doc.write(reader.ReadToEnd());
                doc.close();
                response1.Close();
                // Loop through the html document elements to get postdata values
                string postdata = ParseDocument(doc);
                
                //
                // create a web request for our page
                //
                WebRequest request = WebRequest.Create(url);
                request.ContentType = "application/x-www-form-urlencoded";
                request.Method = "POST";
                ((HttpWebRequest)request).Referer = url;

                //
                // write the post data
                //
                
                using (Stream stream = request.GetRequestStream())
                {
                    stream.Write(Encoding.UTF8.GetBytes(postdata), 0, Encoding.UTF8.GetByteCount(postdata));
                    stream.Flush();
                }

                //
                // get the response
                //
                WebResponse response = request.GetResponse();

                using (FileStream dest = new FileStream(GetFilename(response), FileMode.Create))
                {
                    using (Stream src = response.GetResponseStream())
                        CopyStream(src, dest);
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("ERROR: {0}", ex.ToString());
            }
        }

        private static string GetFilename(WebResponse response)
        {
            
            //
            // try and get a file name from Content-Disposition
            //
            string filename = response.Headers["Content-Disposition"];
            if (filename != null)
            {
                string[] tokens = filename.Split(';');
                if (tokens.Length == 2)
                {
                    tokens = tokens[1].Split('=');
                    if (tokens.Length == 2)
                        filename = tokens[1];
                    else
                        filename = null;
                }
                else
                    filename = null;
            }

            //
            // if that didn't work; try to get it from Content-Type
            //
            if (filename == null)
            {
                filename = response.Headers["Content-Type"];
                if (filename != null)
                {
                    string[] tokens = filename.Split(';');
                    if (tokens.Length == 2)
                    {
                        tokens = tokens[1].Split('=');
                        if (tokens.Length == 2)
                            filename = tokens[1];
                        else
                            filename = null;
                    }
                    else
                        filename = null;
                }
            }

            //
            // if there's still no file name then make one up
            //
            if (filename == null)
                filename = "ding.dat";

            return "c:\\" +  filename;
        }

        private static void CopyStream(Stream src, Stream dest)
        {
            byte[] buffer = new byte[1024];
            int read = -1;

            while (read != 0)
            {
                read = src.Read(buffer, 0, buffer.Length);
                if (read != 0)
                    dest.Write(buffer, 0, read);
            }
        }

        public string ParseDocument(IHTMLDocument2 doc)
        {
            string postdata = "_qf__mainform=";

            foreach (mshtml.IHTMLElement pageElement in doc.all)
            {
                string id = String.IsNullOrEmpty(pageElement.id) ? " " : pageElement.id;

                if (id == "selected_vintage_dates")
                {
                    string[] x;
                    x = pageElement.innerText.Trim().Split(new Char[] {' '});
                  
                    for (int i = 0; i < x.Length; i++)
                    {
                        postdata += "&selected_vintage_dates%5B%5D=" + x[i];
                    }
                    break;
                }
           }
           postdata += "&entered_vintage_dates=&file_type=1&file_format=xls&download_data=Download+Data";
           return postdata;
        }


    }
</code>

hi,
can any body help me while posting the difference between .net framework 1.1 and 2.0?

Thanks

--Krish
prfkrushna@gmail.com

Many differences.

1- Partial classes in .NET 2.0
2- Generics in 2.0
3- Many more classes.

Check this[^] to get an idea

All C# applications should call Application.Quit(); in the beginning to avoid any .NET problems.- Unclyclopedia

My Website | Ask smart questions

hi

i feel it's not that much enough in the interview point of view.
it's better u pls help me some more extent.

otherwise if any written document is there, pls send it to my mail id:

prfkrushna@gmail.com

Thanks

--Krish
prfkrushna@gmail.com

0.9

"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001

Now lets play a game of Find the egg.

Deja View - the feeling that you've seen this post before.

IIRC a couple of days ago the egg was in the middle of a big red circle, part of a traffic
control system. Not sure it is still there though.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

this months tips:
- use PRE tags to preserve formatting when showing multi-line code snippets
- before you ask a question here, search CodeProject, then Google

"They seek it here. They seek it there. They seek that egg everywhere." This poem comes to you courtesy of John Simmon's YerAnus.;)

Deja View - the feeling that you've seen this post before.

Luc Pattyn wrote:
the egg was in the middle of a big red circle

If there was a hailstorm, it would have got damaged right?

Vasudevan Deepak Kumar
Personal Homepage
Tech Gossips

Yesterday is a canceled check. Tomorrow is a promissory note. Today is the ready cash. USE IT.

Don't know, it could have been hard boiled or so.

Luc Pattyn [Forum Guidelines] [My Articles]

this months tips:
- use PRE tags to preserve formatting when showing multi-line code snippets
- before you ask a question here, search CodeProject, then Google

Wait for me.... I ran out of chalk!

led mike wrote:
I ran out of chalk!

You can borrow mine. I've used it to mark out the sacrifice area.

Deja View - the feeling that you've seen this post before.

Good one!

Vasudevan Deepak Kumar
Personal Homepage
Tech Gossips

Yesterday is a canceled check. Tomorrow is a promissory note. Today is the ready cash. USE IT.

"You posted your question here, indicating that you're so thick you couldn't find the VB Forum if it was licking your bung hole." - John Simmons / outlaw programmer

http://msdn2.microsoft.com/en-us/library/t357fb32.aspx[^]

Breaking Changes in .NET Framework 2.0:
http://msdn2.microsoft.com/en-us/netframework/aa570326.aspx[^]

Vasudevan Deepak Kumar
Personal Homepage
Tech Gossips

Yesterday is a canceled check. Tomorrow is a promissory note. Today is the ready cash. USE IT.

Ahhh - you went and spoiled all the fun.;P

Deja View - the feeling that you've seen this post before.

Hi,

We are a French compagny, we need to find freelance compétence in paris to help us with the projects that we have with our customers.
Dont hésitate to contact me as soon as possible.

Hicham Sipkin
S.G.U.I
61 bld National
92250 La Garenne Colombes
email : hicham.sipkin@sgui.fr
Tél : + 33 1 47 69 60 02
Fax : + 33 1 47 69 60 10
portable : 06 80 64 45 14
http://www.sgui.fr
http://www.sguionline.com
http://www.pasa.fr

Bonsoir,
Je suis tombé par hasard d'un surf sur votre site web. Je suis à la recherche de compétence pour intervenir très rapidement sur de petits developement sharepoint que nous effectuons chez nos client ( intranet note de frais, gestion de congés, etc...) Notre dévelopeur est complétement submergé par les projets en cours. Nous serions interressé par des interventions rapide dans les jours ou semaine qui arrivent. Voir une mise en relations pour les futurs projets d'intranet.
Si jamais vou

This isn't the forum for this. If you want to advertise for staff, consider using this[^] forum.

Deja View - the feeling that you've seen this post before.