Need support or guidance? Reach out to @cwensel https://xitter.cf/t.co/5YnPV67z39
Joined November 2010
- Tweets778
- Following10
- Followers1.5K
- Likes7
Cascading retweeted
Do you miss meetups of the Big Data era, when you could learn about RAFT protocol or map-side join over pizza and beer? Us too! My friend @cwensel has launched SF Distributed Systems Meetup.
First meetup is next week, with talks by me and @conor_power23 lu.ma/t6r4mi4v
Cascading retweeted
I always suspected that Parquet and Orc weren't very good, but TIL from @ViktorLeis that one can do much much better with BtrBlocks. cs.cit.tum.de/fileadmin/w00c…
Cascading retweeted
Cascading 4.5.1 was just released to Maven Central.
buff.ly/3YgQHyh
This release includes minor bug fixes and dependency updates.
buff.ly/45qUnAf
Cascading retweeted
Before I worked at Hortonworks putting Calcite in Hive, I worked on Lingual, a SQL interface to @cascading. That was the first commercial use of Calcite (then called Optiq) and some much needed consulting $$
Cascading retweeted
I'm hanging my shingle out again.
I have some new insights I want time to work out into a developer tool for cloud data engineers, but I want to bootstrap it with contract/consulting revenue.
This is how I started @Cascading in 2007.
chris.wensel.net
Please boost!
Cascading retweeted
Scala folks: if there are open source GitHub Twitter libraries you like, you might want to make a clone/fork now. Seems like we are in a situation where anything can happen. – at Paia, HI
Cascading retweeted
really interested in adding mergeable summary as a datatype to @cascading. think having daily/hourly aggregate tables, then daily/hourly performing a last N day/hour aggregation against the partials. cost efficient way to have rolling percentiles/aggregates
Cascading retweeted
cross your fingers, we might have a @cascading 4.1 release tonight, and a quick 4.5 release a day later w/ Hadoop 3 support. Both will include native @ApacheParquet support to replace the support dropped by Parquet. Plus some well worn enhanced Parquet support in 4.6
Cascading retweeted
For fun I really want to fork the @cascading local mode planner to support RocksDB as the backend for joins instead of the memory only hashmaps. Local mode was never intended to scale in joins.
Cascading retweeted
finally @cascading has Apache Hadoop 3 (and Tez 0.10) support thanks to the team at @Foursquare Labs! github.com/cwensel/cascading…
Cascading retweeted
Google gs connector does do aligned block reads...both that and this one work well for backwards seeks (footers, stripe footers -> stripes)
/2
Cascading retweeted
Improving efficiency and reducing runtime using S3 read optimization by @PinterestEng link.medium.com/KGRST76OBjb << do you have a link to the git repo?
Cascading retweeted
For those interested in such things, just pushed cascading-avro packages to github. This gives Cascading 4.x Avro initial support.
github.com/cwensel/cascading…
Cascading retweeted
Ok, one more step away from JCenter, Cascading 4 WIP release packages are now publishing to GitHub github.com/cwensel?tab=packa…
Cascading retweeted
Take note folk. The hope is to make this transparent to builds. Please update your Maven coordinates and test it out.
For reasons, we need to migrate our Maven repo Conjars.org over to a new site and domain to maintain its long term availability. For more information, see conjars.wensel.net/ and groups.google.com/g/cascadin… for the discussion.
Cascading retweeted
Quidem is a testing framework where every script generates itself if it succeeds, generates what it should have been if it fails. No more juggling reference logs; the script is the reference log.
Cascading retweeted
Well crap. Just start migrating everything to bintray and then they decide to shut it down. I’ve been hosting my own maven repo for over 10 years. Now I remember why.
Cascading retweeted
Slowly getting the @Cascading pipeline back up... github.com/cwensel/cascading…, which are landing in bintray.com/wensel/wip and publishing to JCenter.