Geekhack Thread Image Downloader

[cooldiscretion]

I'm not sure if this is the right category to post this, but if it's not maybe someone can move it.


Over the last few weeks I wrote a Python script that will download all images in a particular thread on Geekhack and generate
a report file with URL links to where the image was first encountered in the thread. It started because I wanted to use all the
images from the 'post your clacks' thread to be randomly displayed for my screensaver at work. Then, as I was looking at the
images that were downloaded, I wanted to know who posted them and where so I modified the script to create a report file.
The script will do it's best to find situations where there are duplicate images and squash them from being downloaded multiple
times. It doesn't download gifs, but it cannot currently distinguish between meme jpegs that have been uploaded or valid pictures.


I am in no way an amazing Python developer as I normally write C for my day to day job, but I've created a Github page with
instructions on how to use the script here:


https://github.com/stevegcarpenter/geekhack_image_downloader


Any feedback on how it might be improved or where the instructions aren't clear enough would be well appreciated. And of course,
I hope it serves the purpose for others that it did for me.

[Computer-Lab in Basement]

will this download all of the gifs?

[retrochick]

don't think so.

[cooldiscretion]

will this download all of the gifs?

It won't download gifs. but the script could be modified to only download gifs or even to specify what types of files to be downloaded I suppose.

[azhdar]

would it work to download from the posts of a user, for example : https://geekhack.org/index.php?action=profile;area=showposts;u=36817

[cooldiscretion]

would it work to download from the posts of a user, for example : https://geekhack.org/index.php?action=profile;area=showposts;u=36817

Unfortunately, it won't currently. A thread is only really comprised of two numbers that are important which I extract from the URL address.

The first number comes after topic= inside the url address. That is the number which identifies the thread itself. And, the second number follows a
period after the topic number - it is the page number. However, the page numbers go in increments of 50, which is why page 1 of a thread should end
int 0 and page 2 would end in 50. I'm guessing support could be added to actually download images from a users posts although that could potentially
be something others wouldn't like for moral reasons of harvesting all data from a particular user.


EDIT: For example, this thread has the topic number 77652 and since this is page 1 it is followed by page number 0.


Hence, URL address would be:
https://geekhack.org/index.php?topic=77652.0


The same strategy could be used for a users posts though. The only unique data in the URL address is the user profile number (mine appears to be 36817)
followed  by the page number of said users posts which start at 0 and increment by 50 each page. If you look at page 2 of my posts, you can see this:
https://geekhack.org/index.php?action=profile;u=36817;area=showposts;start=50


This same format can be used to access all pages including page 1, just modify the URL to be:
https://geekhack.org/index.php?action=profile;u=36817;area=showposts;start=0

[trenzafeeds]

This should probably be in suggestions/improvements, or whatever that subforum is called.

[trizkut]

This should probably be in suggestions/improvements, or whatever that subforum is called.

Why? This has nothing to do with modifying GH in any way

[trenzafeeds]

This should probably be in suggestions/improvements, or whatever that subforum is called.

Why? This has nothing to do with modifying GH in any way

I know... it just seems out of place here. Doesn't really matter at all though!  Quite a cool script.

Actually now that I think about it, it could go in MST. Again, totally not an issue though, I don't actually care lol.

[cooldiscretion]

I just updated the script to accept a start and end page to download images.  This means people won't be forced to download all images from a particular thread
when they are just interested in the last 20 pages or so.


To all you lurkers and active members alike, you should use this over the holidays and tell me about any problems or improvements you like. If people really like the idea of downloading
images from a particular users posts, I can add that feature too.