geekhack
Site Announcements and Feedback => Announcements/Feedback/Suggestions => Topic started by: _rubik on Thu, 16 November 2017, 09:27:02
-
I'm not sure if this is the right place for this, but I was wondering if there is a place to get ONLY the live group buys. I have noticed that a lot of threads gumming up the IC/GB forums with products that are closed or past. I'm wondering if I'm missing a specific page or thread etc... and, if there is not, I could very easily throw together a website that acts as a list of live buys.
-
I'm not sure if this is the right place for this, but I was wondering if there is a place to get ONLY the live group buys. I have noticed that a lot of threads gumming up the IC/GB forums with products that are closed or past. I'm wondering if I'm missing a specific page or thread etc... and, if there is not, I could very easily throw together a website that acts as a list of live buys.
Something like this (https://geekhack.org/index.php?topic=57761.0)?
-
I'm not sure if this is the right place for this, but I was wondering if there is a place to get ONLY the live group buys. I have noticed that a lot of threads gumming up the IC/GB forums with products that are closed or past. I'm wondering if I'm missing a specific page or thread etc... and, if there is not, I could very easily throw together a website that acts as a list of live buys.
Something like this (https://geekhack.org/index.php?topic=57761.0)?
Beat me to it lol
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
Hmm. Certainly a good idea, I'd be all for that. I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough. Now, getting everyone to follow the standard...well, we can hope. :))
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
Hmm. Certainly a good idea, I'd be all for that. I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough. Now, getting everyone to follow the standard...well, we can hope. :))
why not just have an rss feed of the GB board that updates hourly or something?
-
Think I’ve got 80 or so current/past buys in my spreadsheet so far. Might expand it and/or get some other contributors to get a lot more, as at the minute I’m only getting details for buys that get bumped.
Would be great to have it automatic, but as Hoff noted, keeping a standard would be impossible. :'(
-
could teach a neural network to do it, but that seems like a lot more work than it's worth
and frankly i don't think anyone here has access to one
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
Hmm. Certainly a good idea, I'd be all for that. I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough. Now, getting everyone to follow the standard...well, we can hope. :))
why not just have an rss feed of the GB board that updates hourly or something?
Because the GB board also has posts from the past -- old things get dug up.
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
Hmm. Certainly a good idea, I'd be all for that. I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough. Now, getting everyone to follow the standard...well, we can hope. :))
I believe I have a clean way to implement this... but it would require OP to drop a comment with a predetermined format. Name, date, quantity, etc. Simple enough. It's just a matter of getting people to post...
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
Hmm. Certainly a good idea, I'd be all for that. I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough. Now, getting everyone to follow the standard...well, we can hope. :))
I believe I have a clean way to implement this... but it would require OP to drop a comment with a predetermined format. Name, date, quantity, etc. Simple enough. It's just a matter of getting people to post...
Shoot me a PM with the details (formatting, etc) and we can work to get it implemented as a "rule". As I mentioned before, I doubt it will have 100% compliance, but myself and the mod team (if they are interested) can go through and modify posts to match the right format. If there was a way to have it know that the format was wrong and alert us, that would be awesome.
I'm not sure how RSS feeds work (or rather, what the possibilities are), but if there's a way to embed it in a post here that would be great. Otherwise you'd be able to link to something that would show the feed, right?
-
I knew you would have an answer to this.
But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.
Hmm. Certainly a good idea, I'd be all for that. I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough. Now, getting everyone to follow the standard...well, we can hope. :))
I believe I have a clean way to implement this... but it would require OP to drop a comment with a predetermined format. Name, date, quantity, etc. Simple enough. It's just a matter of getting people to post...
Shoot me a PM with the details (formatting, etc) and we can work to get it implemented as a "rule". As I mentioned before, I doubt it will have 100% compliance, but myself and the mod team (if they are interested) can go through and modify posts to match the right format. If there was a way to have it know that the format was wrong and alert us, that would be awesome.
I'm not sure how RSS feeds work (or rather, what the possibilities are), but if there's a way to embed it in a post here that would be great. Otherwise you'd be able to link to something that would show the feed, right?
I'll do some tinkering in the next few days. Most likely it'll be a separate website because RSS feeds only format data on the forum to be viewed (and not to post anything new). If GH had an Rest API where I can post via a HTTP request, I could get a thread together... but I'm pretty sure we don't. My bad if that just sounds like a mess of confusing acronyms. Maybe an admin would know about our API status?
Edit: I just tinkered a little with our current RSS feed... it's very sparse and truncated. By that I mean, it doesn't retain much in the way of previous posts, and it doesn't organize the threads in any special way.. It's all just one big mixing pot. Still doable, just a little more cumbersome.
-
This may be considered spam but...
---
GB: Test60
Website: testpost.com
From: Dec 1 2017
Until: Dec 25 2017
---
-
This may be considered spam but...
---
GB: Test60
Website: testpost.com
From: Dec 1 2017
Until: Dec 25 2017
---
Dates in ISO format would be better - yyyy-mm-dd
-
This may be considered spam but...
---
GB: Test60
Website: testpost.com
From: Dec 1 2017
Until: Dec 25 2017
---
Dates in ISO format would be better - yyyy-mm-dd
Dates in ISO are easy enough to do. My bigger concern at the moment is a structure that can be parsed regardless of white space, but at the same time be human readable and intuitive. Yaml is clearly not it.
For anyone technically minded following this: the RSS feed truncates post data and provides 0 info on the OP, thread etc (to be expected). What I can get from RSS is the comment ID and a thread URL, which allows me to scrape any info from the thread I need.