crwlr.software (@crwlrsoft) header
crwlr.software (@crwlrsoft) profile picture

PHP composer packages for web scraping and crawling tasks.

Linz, Österreich
Joined February 2018
Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)Photo by crwlr.software (@crwlrsoft)
I just created a bluesky account for crwlr => bsky.app/profile/crwlr.bsky.… See you there! 🙂👋
26
Trying PHP 8.4 for the first time today. Already love the new DOM features. Especially natively having CSS Selectors ☺️🫶 I'll also immediately start to make all @crwlrsoft packages fit for PHP 8.4! 💪
1
2
100
Package installs are slowly but steadily increasing. Reached some milestones: * 40.000 => crwlr/url * 10.000 => crwlr/robots-txt * 6.000 => crwlr/crawler 💪🏻
2
34
Shipped two bugfix releases yesterday and today: github.com/crwlrsoft/crawler… Thx to Viktor for the PR! This marks the 26th release this year! Also: nice number of github stars! 🤩😅
34
Already added the first new feature in v2: Post browser navigate hooks. When using the headless browser for loading, you can now interact with the browser after a page was loaded, before reading the state of the HTML source. crwlr.software/packages/craw… 🚀💪
1
101
It’s release day! 🥳 v2.0 of the crawler package is finally out! 🚀💪🏻 A lot of work went into removing deprecated 1.x features. Now we can focus on new features and improvements again! Check out the detailed upgrade guide here: crwlr.software/packages/craw…
1
1
152
We just reached 5,000 installs of the crwlr/crawler package! 🥳🚀 Super excited for the next 5,000 - this time, we’ll get there a lot faster! 💪🏻 Also: v2.0 is just around the corner! Stay tuned! 👀
1
3
81
Yay 🥳🍾 we just made it to 💯 stars ⭐️ on github with the url package! 🤩 Thanks everyone for the support! 🙏🏻 If you haven't checked it out yet: it's like a swiss army knife for dealing with URLs! crwlr.software/packages/url/…
Hey Twitter 👋 Please help, I'm just one star ⭐️ away from 💯 on my url package on github! 😅🙏 It’s pretty useful: it extracts the registrable domain, parses internationalized domain names, and more! Check it out: github.com/crwlrsoft/url
1
48
Since the crwlr library uses Generators to minimize memory usage and you also need them when writing custom steps, I’ve written a Quickstart Tutorial. 🤓✍️📗 If you’re new to Generators, check it out - it’s a very useful PHP feature in some situations. crwlr.software/blog/php-gene…
1
3
109
Great news! v1.8 of the crawler package is out now! 🥳🚀 This release brings improved functionality for composing crawling result data, leading to better performance, minimized memory usage, and a more intuitive user experience. Read more: crwlr.software/blog/crawler-…
1
138
Time for a little celebration, the crwlr project reached some milestones! The crawler package finally got 300 stars on github and url is close to 100 and also just crossed 30000 installs. Thanks to bref, the query-string package already crossed a million installs 🤯
1
1
1
97
And also the crwlr.software website is doing better and better in the google search 🥳 Thanks to everyone using the packages! 🙂🫶
27
So cool to see PHP version evolution in installs of crwlr/url. Packagist 🫶!
2
61
Working hard to improve the crawler library! Already 11 releases this year! 💪🚀 github.com/crwlrsoft/crawler… Also had an idea for a bigger change recently, that could make it a lot easier to use. It's about composing results which a lot of people find very complicated.
1
1
1
126
I will try to implement it as a new feature in v1 and if it works, give it some time and maybe eventually deprecate the current way of doing it.
30