Author Topic: [Question] Complete list of live group buys?  (Read 1129 times)

0 Members and 1 Guest are viewing this topic.

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
[Question] Complete list of live group buys?
« on: Thu, 16 November 2017, 09:27:02 »
I'm not sure if this is the right place for this, but I was wondering if there is a place to get ONLY the live group buys. I have noticed that a lot of threads gumming up the IC/GB forums with products that are closed or past. I'm wondering if I'm missing a specific page or thread etc... and, if there is not, I could very easily throw together a website that acts as a list of live buys.

Offline HoffmanMyster

  • HOFF, MAN OF MYSTERY
  • * Senior Moderator
  • Posts: 10355
  • Location: WI
Re: [Question] Complete list of live group buys?
« Reply #1 on: Thu, 16 November 2017, 09:28:21 »
I'm not sure if this is the right place for this, but I was wondering if there is a place to get ONLY the live group buys. I have noticed that a lot of threads gumming up the IC/GB forums with products that are closed or past. I'm wondering if I'm missing a specific page or thread etc... and, if there is not, I could very easily throw together a website that acts as a list of live buys.

Something like this?

Offline Cocopah

  • Posts: 258
  • Location: Socal
  • Lurker Extraordinaire
Re: [Question] Complete list of live group buys?
« Reply #2 on: Thu, 16 November 2017, 09:29:21 »
I'm not sure if this is the right place for this, but I was wondering if there is a place to get ONLY the live group buys. I have noticed that a lot of threads gumming up the IC/GB forums with products that are closed or past. I'm wondering if I'm missing a specific page or thread etc... and, if there is not, I could very easily throw together a website that acts as a list of live buys.

Something like this?

Beat me to it lol

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
Re: [Question] Complete list of live group buys?
« Reply #3 on: Thu, 16 November 2017, 11:26:24 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Offline HoffmanMyster

  • HOFF, MAN OF MYSTERY
  • * Senior Moderator
  • Posts: 10355
  • Location: WI
Re: [Question] Complete list of live group buys?
« Reply #4 on: Fri, 17 November 2017, 19:52:26 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Hmm.  Certainly a good idea, I'd be all for that.  I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough.  Now, getting everyone to follow the standard...well, we can hope.  :))

Offline matt2dlg

  • Posts: 176
  • Location: Oahu, Hawaii
  • Shipping fees so high in paradise
Re: [Question] Complete list of live group buys?
« Reply #5 on: Sun, 19 November 2017, 22:00:39 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Hmm.  Certainly a good idea, I'd be all for that.  I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough.  Now, getting everyone to follow the standard...well, we can hope.  :))

why not just have an rss feed of the GB board that updates hourly or something?

Offline xondat

  • i'm not a star
  • * Maker
  • Posts: 3466
  • Location: United Kingdom
Re: [Question] Complete list of live group buys?
« Reply #6 on: Sun, 19 November 2017, 22:22:08 »
Think Iíve got 80 or so current/past buys in my spreadsheet so far. Might expand it and/or get some other contributors to get a lot more, as at the minute Iím only getting details for buys that get bumped.

Would be great to have it automatic, but as Hoff noted, keeping a standard would be impossible. :'(

Offline Puddsy

  • Foreman of the salt mine
  • * Exalted Elder
  • Posts: 6516
  • Location: The Enigma
  • Can't handle the heat? Stay out of the kitchen.
Re: [Question] Complete list of live group buys?
« Reply #7 on: Sun, 19 November 2017, 23:09:38 »
could teach a neural network to do it, but that seems like a lot more work than it's worth

and frankly i don't think anyone here has access to one
QFR (62g blacks) | MJ2 TKL | Realforce 87U | G81-3000 "Schumiboard" (currently disassembled) | Varmilo Bulgogiboard (Keycon 104) | KBD75 (MX silent blacks) | MIRA (soon!) | Weaven (post xmas, will be MOD-M) | "The Wedge"

ahh, thanks puddsy. can always count on your snarky informitism

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
Re: [Question] Complete list of live group buys?
« Reply #8 on: Mon, 20 November 2017, 11:42:42 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Hmm.  Certainly a good idea, I'd be all for that.  I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough.  Now, getting everyone to follow the standard...well, we can hope.  :))

why not just have an rss feed of the GB board that updates hourly or something?

Because the GB board also has posts from the past -- old things get dug up.

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
Re: [Question] Complete list of live group buys?
« Reply #9 on: Mon, 20 November 2017, 11:47:48 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Hmm.  Certainly a good idea, I'd be all for that.  I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough.  Now, getting everyone to follow the standard...well, we can hope.  :))


I believe I have a clean way to implement this... but it would require OP to drop a comment with a predetermined format. Name, date, quantity, etc. Simple enough. It's just a matter of getting people to post...

Offline HoffmanMyster

  • HOFF, MAN OF MYSTERY
  • * Senior Moderator
  • Posts: 10355
  • Location: WI
Re: [Question] Complete list of live group buys?
« Reply #10 on: Mon, 20 November 2017, 18:28:23 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Hmm.  Certainly a good idea, I'd be all for that.  I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough.  Now, getting everyone to follow the standard...well, we can hope.  :))


I believe I have a clean way to implement this... but it would require OP to drop a comment with a predetermined format. Name, date, quantity, etc. Simple enough. It's just a matter of getting people to post...

Shoot me a PM with the details (formatting, etc) and we can work to get it implemented as a "rule".  As I mentioned before, I doubt it will have 100% compliance, but myself and the mod team (if they are interested) can go through and modify posts to match the right format.  If there was a way to have it know that the format was wrong and alert us, that would be awesome

I'm not sure how RSS feeds work (or rather, what the possibilities are), but if there's a way to embed it in a post here that would be great.  Otherwise you'd be able to link to something that would show the feed, right?

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
Re: [Question] Complete list of live group buys?
« Reply #11 on: Mon, 20 November 2017, 21:49:48 »
I knew you would have an answer to this.

But because I'm lazy, I'm wondering if there is a way to automate this process. Meaning, I'm wondering if we can work a standard into the posts for an RSS scraper to notice. That way, we could compile a list of GBs past, present, and future without needing to manually update.

Hmm.  Certainly a good idea, I'd be all for that.  I don't have the skills necessary for this, but if anyone wants to come up with a way to scrape the info (name, deadline, photos, link) we can implement a standard easily enough.  Now, getting everyone to follow the standard...well, we can hope.  :))


I believe I have a clean way to implement this... but it would require OP to drop a comment with a predetermined format. Name, date, quantity, etc. Simple enough. It's just a matter of getting people to post...

Shoot me a PM with the details (formatting, etc) and we can work to get it implemented as a "rule".  As I mentioned before, I doubt it will have 100% compliance, but myself and the mod team (if they are interested) can go through and modify posts to match the right format.  If there was a way to have it know that the format was wrong and alert us, that would be awesome

I'm not sure how RSS feeds work (or rather, what the possibilities are), but if there's a way to embed it in a post here that would be great.  Otherwise you'd be able to link to something that would show the feed, right?

I'll do some tinkering in the next few days. Most likely it'll be a separate website because RSS feeds only format data on the forum to be viewed (and not to post anything new). If GH had an Rest API where I can post via a HTTP request, I could get a thread together... but I'm pretty sure we don't. My bad if that just sounds like a mess of confusing acronyms. Maybe an admin would know about our API status?

Edit: I just tinkered a little with our current RSS feed... it's very sparse and truncated. By that I mean, it doesn't retain much in the way of previous posts, and it doesn't organize the threads in any special way.. It's all just one big mixing pot. Still doable, just a little more cumbersome.
« Last Edit: Mon, 20 November 2017, 22:19:14 by _rubik »

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
Re: [Question] Complete list of live group buys?
« Reply #12 on: Mon, 20 November 2017, 22:59:46 »
This may be considered spam but...

---
GB: Test60
Website: testpost.com
From: Dec 1 2017
Until: Dec 25 2017
---

Offline rowdy

  • HHKB Hapster
  • * Erudite Elder
  • Posts: 20282
  • Location: melbourne.vic.au
  • Missed another sale.
Re: [Question] Complete list of live group buys?
« Reply #13 on: Tue, 21 November 2017, 19:45:54 »
This may be considered spam but...

---
GB: Test60
Website: testpost.com
From: Dec 1 2017
Until: Dec 25 2017
---

Dates in ISO format would be better - yyyy-mm-dd
"Because keyboards are accessories to PC makers, they focus on minimizing the manufacturing costs. But thatís incorrect. Itís in HHKBís slogan, but when Americaís cowboys were in the middle of a trip and their horse died, they would leave the horse there. But even if they were in the middle of a desert, they would take their saddle with them. The horse was a consumable good, but the saddle was an interface that their bodies had gotten used to. In the same vein, PCs are consumable goods, while keyboards are important interfaces." - Eiiti Wada

NEC APC-H4100E | Ducky DK9008 Shine MX blue LED red | Ducky DK9008 Shine MX blue LED green | Link 900243-08 | CM QFR MX black | KeyCool 87 white MX reds | HHKB 2 Pro | Model M 02-Mar-1993 | Model M 29-Nov-1995 | CM Trigger (broken) | CM QFS MX green | Ducky DK9087 Shine 3 TKL Yellow Edition MX black | Lexmark SSK 21-Apr-1994 | IBM SSK 13-Oct-1987 | CODE TKL MX clear | Model M 122 01-Jun-1988

Ị̸͚̯̲́ͤ̃͑̇̑ͯ̊̂͟ͅs̞͚̩͉̝̪̲͗͊ͪ̽̚̚ ̭̦͖͕̑́͌ͬͩ͟t̷̻͔̙̑͟h̹̠̼͋ͤ͋i̤̜̣̦̱̫͈͔̞ͭ͑ͥ̌̔s̬͔͎̍̈ͥͫ̐̾ͣ̔̇͘ͅ ̩̘̼͆̐̕e̞̰͓̲̺̎͐̏ͬ̓̅̾͠͝ͅv̶̰͕̱̞̥̍ͣ̄̕e͕͙͖̬̜͓͎̤̊ͭ͐͝ṇ̰͎̱̤̟̭ͫ͌̌͢͠ͅ ̳̥̦ͮ̐ͤ̎̊ͣ͡͡n̤̜̙̺̪̒͜e̶̻̦̿ͮ̂̀c̝̘̝͖̠̖͐ͨͪ̈̐͌ͩ̀e̷̥͇̋ͦs̢̡̤ͤͤͯ͜s͈̠̉̑͘a̱͕̗͖̳̥̺ͬͦͧ͆̌̑͡r̶̟̖̈͘ỷ̮̦̩͙͔ͫ̾ͬ̔ͬͮ̌?̵̘͇͔͙ͥͪ͞ͅ

Online _rubik

  • Thread Starter
  • Posts: 322
  • Location: 192.168.x.x
  • Typing on: RF87u + BKE Redux Heavies
Re: [Question] Complete list of live group buys?
« Reply #14 on: Tue, 21 November 2017, 21:10:22 »
This may be considered spam but...

---
GB: Test60
Website: testpost.com
From: Dec 1 2017
Until: Dec 25 2017
---

Dates in ISO format would be better - yyyy-mm-dd

Dates in ISO are easy enough to do. My bigger concern at the moment is a structure that can be parsed regardless of white space, but at the same time be human readable and intuitive. Yaml is clearly not it.

For anyone technically minded following this: the RSS feed truncates post data and provides 0 info on the OP, thread etc (to be expected). What I can get from RSS is the comment ID and a thread URL, which allows me to scrape any info from the thread I need.