4

I have been using Scrapy with Python for web scraping, is there anything similar for Crytal? I tried to make a Google search but I found nothing.

Caffeinatedwolf
  • 1,217
  • 3
  • 20
  • 26
  • 1
    It doesn't seem like there's any in the `awesome-crystal` list [here](https://github.com/veelenga/awesome-crystal). You can ask for one on the `crystal-libraries-needed` repository [here](https://github.com/crystal-community/crystal-libraries-needed). I personally have no idea how hard it is to make a web scraping library, so I'm not sure how achieveable it is to just make the library itself. Hopefully this helps. – Qwerp-Derp Aug 26 '18 at 04:15
  • 2
    As an aside, this type of question generally isn't accepted on SO where you ask for a certain library that achieves a purpose, since it attracts opinionated answers rather than subjective answers. You could ask the same question in the project's Gitter chat [here](https://gitter.im/crystal-lang/crystal). – Qwerp-Derp Aug 26 '18 at 04:19
  • You could download the files yourself: https://stackoverflow.com/questions/42580251/crystal-lang-how-to-get-binary-file-from-http but that's not a scraping library per se. – rogerdpack Aug 29 '18 at 17:52

1 Answers1

5

This site is the best place to look for shards, IMHO. About scrapping or parsing html I started playing with Myhtml but also found Crystagiri and June

MegaTux
  • 1,591
  • 21
  • 26