Complementing both the manual for the Net::HTTP class and Rail’s guide to rendering proxied pages (from the Rails wiki) here is a short script, which makes use of a custom made HTTPS POST query (btw. HTTP POST is basically the same, you just have to remove the ssl declaration and change the destination port).
First, the script makes a GET query to the target host (in the example, it is the login page for Wirtualna Polska webmail). In the next step it reads which cookies the website wants to set. After that, we can make our POST query. We send both filled in POST parameters (and hidden inputs too) and all of the cookies, which would normally be set in the browser. After that the script just prints out response headers and data (if there’s any).
The solution is simple, short and clean, although it took me few hours to find out that POST parameters must be a &-separated string and that the headers is a hash array. After that, it was a piece of cake.
Anyway, here’s the script:
#!/usr/bin/ruby
require 'net/http'
require 'net/https'
http = Net::HTTP.new('profil.wp.pl', 443)
http.use_ssl = true
path = '/login.html'
# GET request -> so the host can set his cookies
resp, data = http.get(path, nil)
cookie = resp.response['set-cookie']
# POST request -> logging in
data = 'serwis=wp.pl&url=profil.html&tryLogin=1&countTest=1&logowaniessl=1&login_username=blah&login_password=blah'
headers = {
'Cookie' => cookie,
'Referer' => 'http://profil.wp.pl/login.html',
'Content-Type' => 'application/x-www-form-urlencoded'
}
resp, data = http.post(path, data, headers)
# Output on the screen -> we should get either a 302 redirect (after a successful login) or an error page
puts 'Code = ' + resp.code
puts 'Message = ' + resp.message
resp.each {|key, val| puts key + ' = ' + val}
puts data
Hey, thanks for this, it’s really useful! Was wondering if you knew of any conveneint way to follow redirects?
It sould be fairly easy to combine my code with the redirect implementation from ruby-doc (scroll down one screen).
Was playing with that for awhile and then I just discovered WWW::Mechanize. Have you seen this thing? It’s totally amazing. It was so easy, I took the liberty of rewriting your script using it like so:
require ‘rubygems’
require ‘mechanize’
agent = WWW::Mechanize.new
agent.user_agent_alias = ‘Mac Safari’
page = agent.get(‘http://profil.wp.pl/login.html’)
form = page.forms[0]
form.fields.find{|f| f.name == ‘login_username’}.value = “blah”
form.fields.find{|f| f.name == ‘login_password’}.value = “blah”
page = agent.submit(form)
puts page.body
So easy! You can also turn on verbose logging by changing that first line to:
agent = WWW::Mechanize.new {|a| a.log = Logger.new(STDERR) }. Plus it does link-clicking and all kinds of other fancy stuff. It’s the best thing since sliced-bread.
Sounds very interesting. I must try it out.
Wow, this turns out to be _extremely_ helpful!!! I WWW::Mechanize takes care of all the dirty work for you. I was able to use it to construct a pseudo-API to a site that doesnt have one
Hi, I am trying to post form data from a jsp page which is using http method GET . How can I use it and get response .
I have the same question as pradeepta : the problem is not how to send data via POST or GET but how to get back the info via POST or GET. I have a form with an input file for a upload in HTML tags. The file is sent by GET method. How do I get the file back to process it ?
@Marc: you need a httpd for that.
Thanks for this. This is exactly what I needed to complete a project. Glad to see the guy who recommended mechanize thanked you first. I prefer this solution because it does not require the installation of yet another gem or plugin. I don’t see where using mechanize for this task would buy me much. Certainly not enough to justify having to manage another gem or plugin,