Dec 04
Peter D Bethke Scraping a form using include_url
Dec 04, 2014; 13:03
Peter D Bethke
Scraping a form using include_url
Hello all,
I=92m working on creating an automated login system which transparently logs a user in to another site behind the scenes first to generate a cookie, so that when the user navigates to that new site, the new site will pick up the cookie and consider that user pre-authenticated.
My strategy is to use either os_process and curl, or include_url to mimic the form parameters that are submitted to the destination site, and grab the cookie data from the response header.
My concern is that there are a number of hidden fields in the target site login page that I think are required to complete the login process. testing using simply the username and password field does not work using curl.
However, if I submit the form and grab all the hidden parameters using firebug, and submit that, it works via curl.
So, I think the optimal solution would be to pre-fetch the hidden and visible parameters and substitute the username and pw (which I can store in clear text or encrypted in a session) along with all the other parameters as scraped from the login page, via curl.
Obviously, I=92m guessing what I=92m doing is similar to how spammers auto submit forms, but in this case its actually legitimate.
Any input would be greatly appreciated. My guess is that some combination of fancy regex to parse the visible and hidden fields in a lasso array, then insert them into a curl command or into include_url would the the ticket. I just have no idea how this might be done.
Best,
Peter Bethke
#############################################################
This message is sent to you because you are subscribed to
the mailing list Lasso Lasso@lists.lassosoft.com
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <Lasso-unsubscribe@lists.lassosoft.com>
Send administrative queries to <Lasso-request@lists.lassosoft.com>
Dec 04
Mason Miller Re: Scraping a form using include_url
Dec 04, 2014; 13:23
Mason Miller
Re: Scraping a form using include_url
Dec 04
Peter D Bethke Re: Scraping a form using include_url
Dec 04, 2014; 13:23
Peter D Bethke
Re: Scraping a form using include_url
Dec 04
Mason Miller Re: Scraping a form using include_url
Dec 04, 2014; 13:31
Mason Miller
Re: Scraping a form using include_url
Dec 04
Bil Corry Re: Scraping a form using include_url
Dec 04, 2014; 19:49
Bil Corry
Re: Scraping a form using include_url
Dec 28
Seth Ganahl Re: Scraping a form using include_url
Dec 28, 2014; 14:27
Seth Ganahl
Re: Scraping a form using include_url
Apr 15
Administrator Re: Scraping a form using include_url
Apr 15, 2015; 11:11
Administrator
Re: Scraping a form using include_url