> Hi
> 
> I was parsing one web page, where data are loaded dynamically when a
> search criterion are given, but no change has been seen in the browser
> url, it is still then "https://www.kleyntrucks.com/trucks/tractorunit/".
> Thus below code is not helpful to get the right data, as per the search
> criterion. Suppose I set the the search field "Matriculation year" as
> 2003 to 2005, and then if you look at the url, you still would see that
> url is "https://www.kleyntrucks.com/trucks/tractorunit/". Thus the
> results are not coming as I am thinking to code.
> 
> How can I handle this situation ?
> 
> require 'open-uri'
> doc =
> Nokogiri::HTML(open("https://www.kleyntrucks.com/trucks/tractorunit/"))
> 
> On the other hand - the
> website(
http://www.flipkart.com/mobiles/samsung~brand/pr?sid=tyy,4io&otracker=hp_nmenu_sub_electronics_0_Samsung
)
> seems good. Now suppose I want to scrap the page, when **Price** has
> been selected between **5001-10000**, then I would get also its
> equivalent url from the browser -
> (http://www.flipkart.com/mobiles/samsung~brand/pr?p
[]=facets.price_range%255B%255D%3DRs.%2B5001%2B-%2BRs.%2B10000&p[]=sort%3Dfeatured&sid=tyy%2C4io&ref=3de01d19-4e68-4ba9-9b3f-9c43497931b0).
> 
> 
> Accordingly I can use this url in below, then all my code will get
> correct data :
> 
> require 'open-uri'
> doc =
> Nokogiri::HTML(open(http://www.flipkart.com/mobiles/samsung~brand/pr?p
[]=facets.price_range%255B%255D%3DRs.%2B5001%2B-%2BRs.%2B10000&p[]=sort%3Dfeatured&sid=tyy%2C4io&ref=3de01d19-4e68-4ba9-9b3f-9c43497931b0))
> 
> How to then proceed with the first
> case(https://www.kleyntrucks.com/trucks/tractorunit/) ? Is there anyway
> ?
> 
> -- 
> Posted via http://www.ruby-forum.com/.
> 

My first thought is that you are going to have to open the browser, 
interact with the page, and then grab the html source.
I would use watir-webdriver but there are other options.

here is an example:


require 'watir-webdriver'

@browser = Watir::Browser.new :chrome
@browser.goto 'https://www.kleyntrucks.com/trucks/tractorunit/'

sleep 2
xpath_matriculation_year = '//*[@id="imprp0"]/div[1]'
@browser.div(xpath: xpath_matriculation_year).click

xpath_beginning_year = '//*[@id="imprp0"]/div[2]/div/div[5]/div[1]/input'
@browser.text_field(xpath: xpath_beginning_year).set 2003

xpath_ending_year = '//*[@id="imprp0"]/div[2]/div/div[5]/div[2]/input'
@browser.text_field(xpath: xpath_ending_year).set 2005

# odd, but needed or the page refresh resets the value of a field you set 
if you don't leave the field
@browser.text_field(xpath: xpath_beginning_year).click

sleep 5
page_html =  @browser.html

^ then use the page_html in nokogiri

Michael
________
Michael Hansen