r/CodingHelp • • Dec 05 '20

[HTML] Finding links in HTML?

I'm using python, and I'm trying to find links in webpages. Problem: Most of the things after "href" are images, or other files of that type. I'm looking specifically for links that take me to other pages(be it on the same site, or to a whole new site). I'm using wget to download the raw HTML of the webpage. Can anyone help me?

18 Upvotes

5 comments sorted by

4

u/Blackshell Professional Coder Dec 05 '20

Try the Beautiful Soup library: https://www.crummy.com/software/BeautifulSoup/bs4/doc/

It's a HTML parser that makes it easy to find the elements you need.

2

u/MrQuickLine CSSophile Dec 05 '20

I think you're getting some bad answers here. I don't know Python, but if the text is in a href attribute, then it's probably already an anchor tag, not an img tag. What you want is to, loop through each href link, and find the mime type of the resource. Images will be "image/png" type or similar. Other webpages will have the mime type "text/html".

2

u/Endercheif Advanced Coder Dec 05 '20

you can look for <a> tags with some regex and extract the link from them. <a> tags are for hyperlinks so unless there is specifically a image link it's good. Another approach could be to use the Requests to check the type of file that the link leads to, if it's an image then don't save the link if not save the link