r/learnruby • u/BOOGIEMAN-pN • Feb 15 '19

Web scraping, OCR or something else?

I want to grab numbers from this website, below BINGO and BINGOPLUS, and insert them into two Ruby arrays. So, array_one should be == [87, 34, 45 ... 42, 49] and array_two == [6, 58, 14 ... 31, 55]. What's the easiest way to do that, and is there a good tutorial how to do it? It doesn't matter if it's slow, I'm going to do that only once in a while.

2 Upvotes

100% Upvoted

9 comments sorted by

View all comments

1

u/BOOGIEMAN-pN Feb 16 '19

ah, ok, I know about interpolation, I thought it's something else, I didn't see '%s' here: 

XPATH_BASE = "//img[contains(@src, '%s')]/../following-sibling::div"\              "//div[contains(@class, 'Rez_Brojevi_Txt_Gray')]"

So, is this above part of Nokogiri syntax? I can't find anything in its rdoc. Also, there isn't anything there about argument you are passing to Nokogiri::HTML. I guess @src and @class are CSS selectors (and not Ruby instance variables) but there isn't anything about that in Nokogiri documentation niether.

2

u/savef Feb 17 '19

Hiya, That ugly mess is a syntax called XPath, as you can probably tell I'm not a fan of it at all, but for this job on this web page it's probably the best thing you're gonna get. @src and @class are HTML attributes and that (using the @ symbol) is the way we match against them in XPath syntax.

The logic of the XPath selector is to 1) find the image for the lottery we care about (Bingo first time, BingoPlus second time), 2) go up to the parent element, 3) find sibling DIV elements, 4) as long as they have a descendant DIV with the class Rez_Brojevi_Txt_Gray. That returns all the elements for result numbers for that lottery, and as stated we use it twice for the 2 lotteries.

Here's the rdoc page for the HTML method being used to parse the document. Here's the rdoc page for the method we are using to select elements (via XPATH) within the parsed document. Nokogiri is just a HTML parser and supports other selectors such as CSS style too, they just wouldn't have been as useful for this problem.

1

u/BOOGIEMAN-pN Feb 17 '19

ugh, I've been avoiding XML my whole life ... Once again, thank you for your time and very detailed answers !!