Jul 27
4scrape Down For Good
I no longer have the time or energy to continue maintaining the 4scrape codebase. There’s currently a crapton of issues with it –
- Cache for searches (potentially just post searches) is broken.
- Threads need to be cached as a whole unit — assembling them from a 500,000-row table is too slow.
- General consistency errors — there’s a bunch of images missing (???)
- The scraper likes to shit itself to keep things lively.
- The backend would occasionally crash/spinlock (???)
- The JavaScript shit is a horrible mess.
I took the entire server offline for now, but I’m going to re-post the shit code here along with a complete dump of the entire database (over a year’s worth of posts, lawds). I don’t really know how to offload the image data (~150GB) — I’ll talk to Shuugo about setting up a torrent (or just turn the web server back on and let people hammer the extra 150mbps).
It was a fun run; I had a good time. Sorry for crapping out on you :(
EDIT: 4scrape source code [178KB]. I should have the full database dump uploaded some time tomorrow (it’s xbox hueg, a little over 500MB compressed comprising almost a million posts and metadata for over 1/3 million images). I’m not quite sure how I’m going to offload the image data yet but it’ll either be over bittorrent, or providing unthrottled HTTP access to the image data (but not the API/front-end, so you’ll need the database dump (or just 1.jpg 2.jpg 3.jpg etc — though you’ll miss *.png etc) to accidentially the whole thing).
EDIT2: And the SQL dump [505MB]. I was running PostgreSQL 8.3; you can probably massage the dump to fit into something else but it uses some Postgres-only features (ie, for the fulltext indexes) so it isn’t going to fit into anything else (ie, MySQL/MSSQL/whatever) as-is. Additionally, I turned back on the image access (for what’s still there) and turned off the access restrictions. Feel free to hammer the balls out of the server while I figure out a better way to get them off.
EDIT3: Go to 4walled.org instead.
454 comments454 Comments so far
So long 4scrape… i’ll miss you :_(
Why not put some money making ads on it. So it’s worth your while.
You will be missed. You provided the best wallpaper service ever. You’re efforts shall not be forgotten. Walk proud, for countless number of computers are now fashionable thanks to you.
There is no justice in the world. R.I.P. 4scrape.
Thanks for everything bro, i hope everyone appreciates the effort you put into this wonderful site, this is very saddening, but you tried you’re best…
be proud
So long, and thanks for all the fish.
NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
Maybe some day it’ll be back?… Maybe? =(
don’t give up 4scrape! I loye you
It was a great tool, and I know a few dozen people who used it.
It’ll be missed. It was really useful.
Wow i never even heard of 4scrape until today, i missed the (nice) boat, ;_;
cant live without 4scrape
one of the best sites in the internetz
If I was an emo, I would have cut myself now :(
Thank you for one of the best pages on the internet.
sure is /emo/ in here
WHYYY?! Must 4scrape die?! It was a wonderful site that gave me A LOT decent wallpapers and ideas on how to customize my desktop. As anon said
“You will be missed. You provided the best wallpaper service ever. You’re efforts shall not be forgotten. Walk proud, for countless number of computers are now fashionable thanks to you.”
And i sir, say thank you for this website. I hope you may find a way to revive 4scrape of its problem and if you do i hope you allow for uploading users to add tags/filter feature to the pics so that way we dont see the same pics over and over again (DIE JEFF GOLDBLUM WALLPAPERS! and many other pics >:( )
Out of curiousity, cant you (hark) leave the code on GoogleCode and ask for users to help you improve the coding?
Dont give up !!!
Dammit was hoping to see this come back on line, well thanks for the good run, my wall paper folder was getting large thanks to this site.
NOOOO! I love 4scrape!! Plz dont die
NOOOOO! I loved 4scrape and would host it myself i could (which i can’t :’( )
On a serious-ish note, why not put all the files into smaller torrents and seed a couple for a time. thats the best way i can see myself of moving the 150Gb’s of data without FedEx.
Nooooo puhlease, dis mai only wun!
It’s a terrible shame to see you give up, I loved 4scrape to death as it helped me search for wallpapers for my various computers by size and even though it was sluggish I’d always find a few new papers to add to my collection. I was hoping to get some wallpapers for my new 1920×1080 monitor by popping by 4scrape, but alas it seems I have missed the ship for good.
I hope you’ll change your mind and pick it back up, You’ve really put some effort behind the site even if it was a massive shitcan of code.
thanks for the great service bro :(
hopefully now that the source is out people will improve it etc though, maybe even get a whole 4scrape clone running~
noooooo this is awful :(
;_;
Post all ur code, someone is bound to start where you left off.
I will miss this, now I have to get walls.
Manually.
Sorry to hear this, I loved 4scrape.
Where the hell am i going to get worthwhile walls now? x[
4scrape was the most amazing idea ever, never knew how many kickass wallpapers there were out there… now its back to fucking keywording google to hell… i hope theres a way back =(
lol haskell
You have done a tremendously great effort. Thank you loads!
wow, wallpaper hunting will never be the same again.
+ imma definatly download that torrent!
4scrapes will be missed =(
>x[
>=(
Sure is weaboo in here
Thanks for your hard work. If you need extra program hands I wouldn’t mind doing some jobs for you.
Sorry to hear the system was such a mess.
You will be missed.
And the world has grown a little darker.
I used to spend hours on 4scrape…I wish m00t would just make a scraper for /wg/ and post a link….
Wow guys, I <3 you too. Going to try to answer the comments I guess.
> Why not put some money making ads on it. So it’s worth your while.
It had ads for about half the time I had it up. They weren’t worth my while — I pulled out about $100 over the past year (ie, not even enough to cover a month’s hosting). That said, taking it down wasn’t as much as a money issue as it was “this thing is falling apart and becoming silly.”
> Out of curiousity, cant you (hark) leave the code on GoogleCode and ask for users to help you improve the coding?
I guess that was a possibility. I don’t think it would have worked very well. First, the code is written in a very inaccessible language, Haskell (where you can’t change a variable, for instanec). A lot of the bigger problems were scaling issues and couldn’t really be fixed with just code (need to get a real text indexer, for example). And to fix the scaling issues there’d have to be some way to distribute the database dumps so people could play with it on their machines.
Way too much overhead for something that was designed to be completely hands-off.
> On a serious-ish note, why not put all the files into smaller torrents and seed a couple for a time. thats the best way i can see myself of moving the 150Gb’s of data without FedEx.
Lawds, it’s going to take some work to get the image data off (having trouble just getting the 500MB compressed database dump off it…) — but I’ll do my best in the hope that something can be spared.
> Post all ur code, someone is bound to start where you left off.
Will update the OP in a minute with a tarball of the source. Attempt #3 to scp the database dump over to my Dreamhost account says it’s halfway done, but has 22 minutes left. I’ll check it in the morning before going to work.
anyway tl;dr <3
this was the best site on the internet :(
fgsfds
Just one thing…
Thanks for a lot of fun..
See’ya… In sometime, somewhere, somehow…
:(
I shall miss 4scrape. <3
R.I.P~
Also sad to hear this, hope someone else takes it on.
:( nooooooo i love you so much 4scrape..
nononononoo you can’t be done :(
No way…Its like Billy Mays died all over again. R.I.P.
I loved you 4scrape.
Rest in peace.
*Heart Breaks* RIP
i will definately be taking the 150 gb of walls. this will definately be missed *tear*
Sad to see 4scrape go :(
Thanks for posting the source code, though. At least I can make my own PC scrape for wallpapers now..
Well, now IQDB is effectively useless. Good fucking job.
Should’ve written that shit in erlang bro. It’d be concurrent as fuck.
I got the sourcecode and I installed python on my server. I am trying to figure it all out. I loved 4scrape enough that I want to try to bring it back from the dead. I don’t know if my server can handle the load, but wth I might as well try. If I can get it goin I will be passing out the site addy.
R.I.P
good night sweet prince
Goodbye 4scrape
shit.. id pay moneys to have 4scrape back.
look into that, set up some paypal shit or something?
theres a high demand for 4scrape there, it was pretty popular, and someones gonna make money with it if you wont.
tl;dr bring it back, i’ll pay $$$ T_T
Wait, where the fuck are the images?
FUCK. jeezus on a cocksickle
im fuckin addicted to that random button!
FUUUCK!
i want my random button!!
dont you die on me 4scrape! nooooooooooo..
Breakin’ my heart man–hope you find a way to make the time and return. Thanks for the awesome work.
Well, shit. It was a good run and I’m thankful for it.
Nooo :(
4scrape was the only worthy wallpaper site in my bookmarks. I’ll just hope the wallpapers will be published somewhere for download. It would be such a waste of time and effort if it went bye-bye… ;(
I’ll miss 4scrape…
you will be missed man
it will be missed man
yea, fuck, I’d be willing to throw some $$ to see 4scrape pull a lazarus. Probably not much by many peoples standards (I’m poor lol), but hey~
You will be missed, by me and many others. Goodbye 4scrape. One day we might meet again…
Meh. It was good for people that don’t go outside the chans or have small wall collections. It was a decent enough source to send people to if they wanted common/popular wallpapers. However it was slow and it had a lot of errors. Sorry you couldn’t fix it.
BAAAAAAWWWWWWWWWWWWWWWWWWWWWWWWWW
Nooooooooooooooo I <3 4scrape :( Hope it comes back to life in some shape or form. Would be willing to donate monies *nods*
My heart is broken :(
> where are the images
They’re on the server waiting to be pulled off (you’ll probably want to import the database dump, then go through the img_full field to build URLs). I’ll have a torrent for them eventually; I’m not quite sure how to wrap up 300,000 files effectively.
> theres a high demand for 4scrape there, it was pretty popular, and someones gonna make money with it if you wont.
I seriously hope someone with more time and resources than I is able to pick up the project and make a couple of dimes off it. Nothing would please me more :)
If someone uses the code and somehow manages to put a (hopefuly) similar site up and running, I hope it becomes popular very quickly so that I won’t have trouble finding out about it.
Anyways, thanks a lot for all the effort you put in keeping 4scrape up. You sir, are awesome.
It was a good run.
The service you provided will be sorely missed.
You can always torrent the image data.
Dude, wtf?
where am i supposed to get my wallpapers now?
i cant go back to /wg/.
Now what about the wallpapers! Is there any other place we can get them?
I So fucking sad about this!
A shame I didn’t use 4scrape more back when it worked.
And you wrote it in Haskell, that’s cool. But even though Haskell is ELEGANT AS FUCK and it’s hard not to want to use it, maybe you should have picked the right tool for the job and used Erlang.
Perhaps you’ll rewrite 4scrape someday?
Also, I didn’t think your blog has that many readers.
4scrape really was a nice service, thanks (´・ω・`)
> Erlang
[rant about dynamic typing in large systems]
I was working on another project in Erlang and ran into so many stupid issues (mostly to blame on the erlang_couchdb library fucking up and the Bayeux protocol specification being retarded) that I canned it.
Web development, in general, is absolute shite.
> Also, I didn’t think your blog has that many readers.
It doesn’t — 4scrape just had that many users :(
You’re right, Haskell’s type system is one of it’s best points, along with its syntax.
I never really used Erlang myself in anything serious, so I don’t know how good it would be for web development, only that it should be easy to create a nice distributed and scalable system. Anyway, you’re the EXPERT WEB DEVELOPER here :)
torrent images in 25gb stacks ?
reason, if some shit bad fuck happenes to one torrent.. you have the other’s to keep pple entertained while you get the orig back :)
> EXPERT WEB DEVELOPER
I died a little on the inside.
> torrent in 25GB stacks
That’s still like 75,000 files in each torrent. Should I tar the files (guh)? What tracker should I use?
I’m trying to build the torrent with ctorrent, but it looks like it’s going to shit itself. At the very least, it’s going to take 6 hours (or more) to finish hashing the files. I estimate tar’ing the files would take about 6 hours as well. Can’t do both at once — they’re both I/O limited (ie, would fight the HDD for access and slow each other down).
lmao.
I’ll have a torrent eventually; if you can’t wait download the database dump, install PostgreSQL and write a 10-line script to fetch all the images over HTTP.
RIP
thanks for the service
Words of thanks are pitiful repayment for the wonderful hours I spent on your site. Truly nothing else like it out there, and I’m deeply saddened to see it go.
Man, I still had some stuff in my favourites that I didn’t save. Well, it’s what I get for being lazy…
RIP 4scrape, you and TV Tropes were the only sites I could have wasted an entire day browsing.
if you can’t wait download the database dump, install PostgreSQL and write a 10-line script to fetch all the images over HTTP.
——
I have a server with over 150GB of free space, and I’ve downloaded the database and installed postgres, etc. Care to share this 10-line script to fetch the images over HTTP?
> Care to share this 10-line script to fetch the images over HTTP?
Seriously? Please be sure to replace the database credentials with your settings. (And I guess I’m sorry for being an asshole with that particular piece of code, but I just couldn’t resist).
Guh, and as a note that script uses ‘pgdb’, which is the PyGreSQL library which you’ll probably need to install separately.
NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
we need you ! for posterity !
I am heartbreak and will miss. Thank you for best wallpaper.
I was kind of hoping for torrents by board, if that’s possible. Since I don’t give a flying fuck about anything on /wg/. (or /hr/ if that data is still around)
> torrents by board
Ugh, that would have been a really good idea, actually. I’ll have to look into doing that.
For reference, the current torrent of everything is 30% done hashing (it’s been running for 6 hours). I’ll have to spend some more time tonight and have it create two separate tar files (one for /w/, one for /wg/) and create two more torrent files or something.
is there a way to convert that script over to mysql? I am thinking I can, but I would rather ask here first.
> is there a way to convert that script over to mysql?
Converting the script isn’t an issue. The main issue is going to be massaging that SQL dump so a MySQL database will actually accept it.
Unless you meant the 4scrape backend over to MySQL. Which is possible, but involves rewriting basically everything which touches the database on the backend (or going through the MySQL ODBC driver which is a massive PITA).
aw… you chibi-human… desudesudesudesu!
thx for everything – especially for my wallpapers from DtBLACK (^w^) – awesome tool it was.
sayonara (T_T)/~~
I’m so fucking happy that the piece of shit is dead
I got one picture of some naked bitch with a snake, and that’s it. It was a shitty picture, too.
I hope you take your fucking Haskells and PostureSquirrels and die in a fire.
http://lolipower.ovh.senshi.jp/forums/index.php/topic,369.0.html
Help this guy bringing life to 4scrape, please :P
His name on the shoutbox is “kamoricks”, so you can even help him in real time, if you can…
Here’s the shoutbox link: http://www.lolipower.org/newchat.html
4scrape: THOU SHALT NOT DIE!
There’s someone already working on bringing 4scrape back up. Looks like he sucks when it comes to Haskel language, so if you could help him, it would be nice.
Just go to lolipower.org -> forums -> wallpapers -> “4scrape is DEAD. For good.”
Wouldn’t it be a easier method to put into smaller 250-500MB torrents therefore allowing a quicker download and easier to seed if you release them 1 at a time as they finish hashing? as when they’ve been downloaded you can stop seeding and let those finished seed.
Too bad I can’t post the direct link to the topic <.<
RIP 4scrape. I hope someone picks up the project and finishes what you started. Well done!
@Anonymous with posturesquirrels,
I lol’d, hard and loud.
Also. I hate to see 4scrape go down, maybe it should have been more community paid for? Ugh, someone said donations to paypal, I say fuck yeah.
Thank you so much for the time and effort put into 4scrape :( I heartily appreciate it.
RIP 4scrape, you will be missed dearly.
Taro <3
:(
Dude you’re awesome, the site was awesome. You made my life awesome with hours of images
Hope that the project will not die and that everyone can enjoy it again
Hate to lose 4Scrape. Frequented it everyday at LEAST 20 times.
You will be missed.
for serious, what am i gonna do with my desktop now?
Lol, I never thought I’d see the day you got 100 comments on your blog
I was wondering, what if you asked TinEye.com to help you out? They are in need of more databases to add to theirs for pictures and since they’re the “new kid on the block” for searching images I think they would be interested enough to look into the issue, Taro (or Hark?).
*same person who asked why not post the code on GoogleCode :p
Thanks for the code in mysql. I am gonna try to do some of it from scratch. I am just more comfortable in MySQL. My hosting package also won’t let me use Postgre. So I have to try some sort of compromise…..if I can do it.
It’d be really nice if you could separate /w/ from /wg/ and the other wallpapers.
Say it aint so….say it aint so…..*sob*
Q_Q It was fun.. Sigh I was getting ready to make a new torrent of my scrape
lol now what the crap do I do with my server… helping with the 4scrape load gave it something to do.. Q_Q
ok so… can someone help me with this sql and source code thing. i didnt do too well in computer science class… add me online and send me an IM to help please and thank you. aaron_paul13@hotmail.com thanks in advance
Wow, that really sucks man, I remember when 4scrape initially debuted. After a while I pretty much started going there almost exclusively, it was alot better than sifting through /wg/ all the time.
Sad to see it go, thanks for keeping it up as long as you did though, it yielded alot of great wallpapers.
So sad to see 4scrape down =[
could someone explain how to use postgre to get the images? thanks.
I will take over this project.
signed Anon/w/mous.
Well, this sucks. Ever since 4chan was down earlier this month or whenever, I can’t get back on it. It’s always slow as hell. I can’t even get an page to load. I guess it’s just me, because my friend who goes there says it’s fine for him.
Fuck my life.
Hope someone will be making a new database soon :D
Noooooooooooooooooooo, pls don’t give up!!
You inspired me to make my own wallpapers site,
somehow you are my mentor.
Still can’t believe you didn’t get rich
with such a great site. Just spam some
advertisement and go for it!
We’ll miss 4scrape!! =(
Thanks for the dealing with the site as long as you did. Found so many amazing wallpapers there. Spent a lot of my downtime at work just browsing through. I really hope someone else pics up the project.
Taro, if the problem with 4scrape is that the codebase is shit, why don’t you rewrite it keeping in mind enterprise, scalable turnkey solutions which will leverage your core skillsets?
> if the problem with 4scrape is that the codebase is shit
Well, one of the bigger problems is that the entire thing needs to be integrated with a real fulltext indexer, and those are almost exclusively turkey solutions.
R.I.P. 4scrape, you shall be missed! :(
Good. I hated 4scrape. If there was one freaking site that kept me from working all damn day, it was 4scrape. Now I can get back to work.
^ Aww, who am I kidding, I’ll never get back to work.
Maybe just disable fulltext search completely? Wallpapers could be searched by their color and/or tags.
Noeeees,digging around das intarwebz for pictures, will be like impossible hope someone takes up the challenge of fixing a new 4scrape
hey there, i really hate to see 4scrape die, i used it quitte a lot and whenever i was in need for some good quality wallpaper i used it i would really love to see stuff come in smaller torrents so i can just get the images you hosted or just to see a way to get the images without “the scraper: being online with search etc.
> Maybe just disable fulltext search completely? Wallpapers could be searched by their color and/or tags.
That means indexing by color and adding support for tagging — which is itself quite an undertaking with the shit codebase. It really needs a rewrite to fully integrate those kinds of features.
Anyway, at least a couple people with servers at OVH have pulled all the images so I imagine *someone* is going to put up an instance.
[...] plutselig meldingen opp som følger: Down for good, source, db dump + images available; refer to http://blog.desudesudesu.or… for details. It was a good run, [...]
I am okay with java but I have a friends that have master degrees in computer science and are java pros. I will do everything I can to help get this site back up. I have obtained of 4gigs of images from this site. If you can get a hold of me, just let me know if you want my email. Even if you don’t want to host the server, I will get it set up myself. I have plenty of hard drive space to spare (over 500GB) and one helluva desktop/server. I go to Michigan Tech University and your site is one of the most popular among the students there. Your efforts in this site are beyond appreciated. I thank you for all that you have done. I hope through the efforts of anon we can get this site back up. 4scrape FTW!!!
I have a question.
About what percent of the images were from which board? How much from /wg/, /hr/ and /w/ individually? Which one took up the most space?
4scrape was the best site ever!
Sad to see it go, but I do and a lot of other people apperiate the effort you put in!
Thanks again!
R.I.P 4scrape
> I am okay with java but I have a friends that have master degrees in computer science and are java pros. I will do everything I can to help get this site back up.
Too bad none of the components are written in Java (shit language, btw). Everything you need to set up your own instance has already been posted above (source, database and access to raw image data).
The reason I took it down was because I didn’t want to put any time into maintaining it. I’m not going to put any time into installing and maintaining an instance elsewhere (thanks for the offer though).
> About what percent of the images were from which board? How much from /wg/, /hr/ and /w/ individually? Which one took up the most space?
I don’t have a measure of disk space (with the exception that /hr/, despite having the least number of images probably takes up more disk than either /w/ or /wg/), but here are the image/post counts for each of the boards:
First off, thanks for releasing your db.
I have something like 4scrape which I’ve had running off and on since spring of last year. Since I haven’t been running it 24/7, there are a lot of gaps (only have ~410k images over the three boards) which I hope to fill with the data from 4scrape.
My question is how long are you going to allow us to keep raping your server via HTTP? I have to figure out how to jerry-rig image data from your PostgreSQL db into my MySQL db.
I can’t really use the torrent since Comcast has a 250GB cap, and a 150GB torrent (with a lot of images I don’t need) takes up a serious chunk of that.
But thanks for everything you’ve done with 4scrape. It was great inspiration for when I worked on mine.
[...] [edit - 4scrape is dead] [...]
beginning batch download of 4scrape, will post 1000 part torrents as i go.
Dude what the hell! Was the best one on the net! :(
*cries*
;_;
:(
dude i hope this is a big hoax, Please don’t go
fuck
So many people would give you cash
get paypal, adverts, BUT DONT LEAVE US ALONE
> The reason I took it down was because I didn’t want to put any time into maintaining it.
Though it’s not like you have anything better to do, right? （ ≖‿≖）
How to you import SQL files into PostgreSQL in Linux? I’ve only ever used MySQL.
Thank you…
im crying :’(
m2 :’(
me 3 :’(
> My question is how long are you going to allow us to keep raping your server via HTTP?
I plan to keep it up until we ditch the server in October, but don’t hold me to that.
> > The reason I took it down was because I didn’t want to put any time into maintaining it.
> Though it’s not like you have anything better to do, right? （ ≖‿≖）
(° – °)
> How to you import SQL files into PostgreSQL in Linux? I’ve only ever used MySQL.
Just pipe the uncompressed dump into the command-line client –
psql -U user -h host -d database < dump.sql
Indeed very sad to see it go. At some point i had a RSS feed on 4scrape just to see a bunch of crap everyday (and luckily some occasional pearls) and to keep me from doing something productive.
BTW i’m the guy who made u the favicon :D
4Scrape will be missed.
:[[
> BTW i’m the guy who made u the favicon :D
Thanks bro, that was an awesome icon :)
there is never enough time! miss you 4scrape……
R.I.P 4Scrape :/
You will be missed.
:((((((((((((((((( R.I.P 4Scrape
R.I.P 4scrape.
Lovely site to find all sorts of wallpapers.
God damn.
I can’t believe this.
It was a good run. Great site, great wallpapers. You will be missed, 4scrape.
the best wp site ever died ;_;
Noooooooooo!!!!
http://www.mediafire.com/download.php?toj2jjlxojv
4scrape.sql.7z – repack (64.67 MB), just though someone would prefer to download less (and it’s extracting faster, i think).
How the fuck do I download ALL of the images from 4scrape? You say it’s 150Gb, and I want it.
A how-to guide or something would be useful.
Is anyone just going to upload the images or what?
nooooo…. goodbye, 4scrape. You were amazing.
I’ll miss you, 4scrape. You were the net’s best site.
FFFFFFFFFFFFFFFUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
seriously i loved that site, shit now i’m gonna have to fucking visit 4chan daily
thanks for ruining my life
y u do this? :(
Q_Q
Sigh I really <3 4scrape!
Cries all night long
here is some stats from my scrape this month
119208 file(s), 28,113.6MB disk used
Thanks for all you’re efforts! 4scrape will be missed, even if it was a huge problem child.
We can rebuild, I have a server at my house. I love PHP/Python/SQL/CGI/Perl. I have 50/50 internetz. Lets fucking do this!
One question: if I mail you an ext. hard drive, would you load the pix so I don’t have to scrape for months?
Two Question: is there any legal issue surrounding ads on this type of site? just askin…
Share teh code man, we need 4scrape
he has the code shared…
If I may suggest the format of the torrent, separate w from wg from hr, then subseparate from normal/risque/tits
That way, I can download all the tits from /wg/ :D
Well, All good things Must come to an end….after going thru almost half of the wallpapers after I found the site, I have to say, it was one of my fav sites on the web. Some of the WPs were crap, some were worse, and some were amazing. It was always worht it to come back and see updates daily. 4scrape, you will be missed.
Well, that sucks, big time. Could you not, as your last act as Master of the Walls, post a guide on how we can put this on our own free hosting (like 000webhost), just for our own private use?
*Goes to search through /wg/*
Thanks for all of your obvious hard work in trying to make it work, and I’m sorry to see it go.
>If I may suggest the format of the torrent, separate w from wg from hr, then subseparate from normal/risque/tits
That would require him to go through every single post and move them to another section, then hash them and would take up alot of taro’s time. When he finally releases i just hope to be able to download them in time with everyone taking the bandwith
http://thepiratebay.org/torrent/5031244/%5B4scrape%5D_desudesudesu_%5B1-1000%5D_JPGS
Why don’t you just opensource the codebase and let someone with both the energy and the time fix the issues?
shit sucks, 4scrape was awesome. thanks for all the work though!
Fucking thank you man.
I discovered 4scrape kinda late, but hell, i loved to waste hours on it.
Once again, Thank You !
No, it’s to good to die!
Someone will take the torch.
please, use opentracker, not the fucking pirate bay
I’m sad to see it go. Goodnight, sweet prince.
>please, use opentracker, not the fucking pirate bay
This. For obvious reasons, thanks for playing though bayfag.
Why the best thing should always die ?
I can’t spend a day on the internet without going to that website,what a big lose, it was the best of the best wallpapers website on the internet, it was more than a wallpapers website, it has something magic, there is no other wallpapers website like it on the web …
I hope you guys will find a way to get it back up or at least upload the wallpapers somewhere else
BAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALLS
I just got back from vacation, and this shit goes down. BAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALLLLLLLLLSSSSSSS
I would gargle balls for this site.
Well… this comes at a shit time. I just reformatted my Ubuntu box. Poor 32″ monitor feels so naked without the 4scrape love.
However… looks like I get to rebuild my Perl image scraper… now that something else isn’t doing it for me… with a pretty cool interface.
Back to the CLI I go.
GOOOOD I MISS YOU! COME BACK!
I miss you, 4scrape.
Thanks for the hard work.
Whenever you post that torrent, I will seed that for fucking ever in the name of 4scrape =(
Much love for 4scrape! No worries, you didn’t crap out; you’re still awesome.
Well this sucks! I spent MANY nights scrolling down for eons and refreshing Safari when the site would lock up to continue my quest for the BEST WALLPAPERS OUT THERE! WTF am I going to do now??? SOMEBODY has to bring this beast to life again, PLEASE!!! WE WANT 4SCRAPE — WE WANT 4SCRAPE — WE WANT 4SCRAPE!!!!!!!!!!!!!!!
OMG I LOVED 4scrape, it was great now /wg/ is the only place for decent walls. Please tell me where the torrent is.
you will be missed
[...] July 27, the creator’s blog said the site is down and he/she won’t cast rez on it. The source code and SQL were posted, [...]
upload more torrents pls =)
For the torrents, can you also include the png files? The previous torrent had the 1000th picture filename but according to the site, only had about 800 files.
fucking motherfucking god….
this was my favourite site in all internet T-T
We need another 4scrape!!!!!
when i get a job im gonna pay a host and try to follow 4scrape’s steps =)
i find this
http://thepiratebay.org/torrent/4907012/meimei-20368Files-May-21-2009.rar
Thanks for keeping 4scrape going as long as you did, it was really a great site.
I’d rather wait for a torrent that this guy actually said his or his friend made, then even if it’s bayfag shit I’ll bother to dl it
Why the fuck can’t he have other people maintain the site instead of doing it all by himself? There’s many people willing to help on this site, and god knows how many projects have started up thanks to this faggot
For the remaining time, put some ads and a donate button on the site.
Salvage what’s left of it.
In the mean time, I’m reeling the images in with a modified version of foo.py, but it got stuck at 29,516.
When it’s done, I’ll host the images on my server, but won’t scrape 4chan or the other chan’s.
Torrents are for fags, and certainly TPB.
To the anonymous above me, could you upload the modified foo.py?
yes that would be nice and at the same time, could you give a quick summary on how to do this? I haven’t even the faintest clue on how to use it
Well, i’ve got my hands on the source codes, i’ll see what i can do. btw fuck the syntax of haskel is complicated, my original plan was to translate it too a more flexible language.
Anon powers has decreased substantially now. :’(
dang it how do I run the foo.py script
This can’t be. SHUT DOWN EVERYTHING
Why Haskell, why? ç_ç
I finally got time to sit and play with the foo script. I got it to run and I am getting an error. I am hoping it’s a simple error. it is:
File “/usr/lib/python2.4/site-packages/MySQLdb/cursors.py”, line 163, in execute
self.errorhandler(self, exc, value)
File “/usr/lib/python2.4/site-packages/MySQLdb/connections.py”, line 35, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1146, “Table ‘db293116618.s_images’ doesn’t exist”)
that has me lost. I made the Mysql database for it and I input all my information for it correctly. I am learning as I go, but this has me stuck.
Oh no I missed the 200GET :<
digglesworth, assuming you are the anonymous that modified the foo.py script and also downloaded 29,516 images, if you upload the script i’d be glad to take a look and see what is wrong.
I am not the one that d/l the 29,516 images. I am the one that wanted the foo file in MySQL. I am trying to understand all this on the fly. If I can get as far as scraping I can go from there and do what I need to do.
http://www.mediafire.com/file/zdj2zmjam4d/foo3.py
there is the script I am using
I’m not saying good bye forever to 4scrape, someone can and will revive the site D:<
I made a batch file that will let you guys download all images from http. I’ll release it when I’m done downloading myself (I’m almost done with /w/ and got 60% of /wg/).
This is a sad day. The end is nigh.
OH HOW THE MIGHTY HAVE FALLEN. GODS FORSAKE US.
<3 4scrape
fucking amazing job man, too bad it came to an end
4scrape WILL be missed, this was the only place I got the best wallpapers for my resolution :’(
Can we please have a Rapidshare (or other file host) upload of all the images? You could make split rar files or just each image separately uploaded with RS uploader. Surely that would be quicker to get all the images off the server, rather than everyone raeping the DB at the same time?
goodnite sweet prince
4scrape will be missed… im willing to donate what little i have in paypal to help bring it back to life
“Can we please have a Rapidshare (or other file host) upload of all the images? You could make split rar files or just each image separately uploaded with RS uploader. Surely that would be quicker to get all the images off the server, rather than everyone raeping the DB at the same time?”
Wow, he said roughly 150gB in image data. How the fuck is he going to upload 1,500 file parts to rapidshare, honestly? A torrent of all the images would suffice.
FML
Hello I wanted to say thanks for your awesome site it was really something that i used a lot its hard to find images that can look good at 1920 x 1200 for free. When my friends would come over they always ask where i get these cool backgrounds i would be selfish for a while and remind them of the web site again. Good work man im truly sad to see it gone.
;_;
“I made a batch file that will let you guys download all images from http. I’ll release it when I’m done downloading myself (I’m almost done with /w/ and got 60% of /wg/).”
I fucking love you man! Hurry up and release it!
shhhhhhhhhhhh
dont let others know
the server will be raped beyond measure once he releases it
Isn’t there anyway to just rollback the server to a previous date when the search cache was working and just disable posting and scraping? At least so we could search whatever images are gathered, 1/3 of a million is enough to keep anyone happy.
“Wow, he said roughly 150gB in image data. How the fuck is he going to upload 1,500 file parts to rapidshare, honestly? A torrent of all the images would suffice.”
Ok, so a friend of mine sorted out the sql stuff and gave me all the image URLs, and my app I wrote is currently downloading them all. I’ll try and get them all sorted, then upload to RS.
>”Isn’t there anyway to just rollback the server to a previous date [...] 1/3 of a million is enough to keep anyone happy.”
If this is possible I’d be all for it. That way it would at least give the world a chance to salvage what they can before all the torrent links come out. On that note, FML I’ll have to set aside a friggin week sorting out all of these walls once I grab a dl link.
Mr. 4scrape don’t leave…get a crew to help you… /b’aw
if someone could give me just ONE link to ONE image then I would be able to get them all but I can’t find that directory and I really want these
So sad to see it go D:
I’m going to torrent all the images *tear*
YOU WERE SO GOOD TO US!
thanks for this site!
WHYYYYYYYYYYYYYYYYYYY?!?!?!?!?!
WRRRYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
is there a site like 4scrape?
I have 1TB virginity asking to defiled by Taro’s 150GB katamari ball of doom.
Bring on the torrent~
…sifting though all that crap is gonna be a b****…a stupid b****…
[...] fuck. I don’t visit the site for 2 weeks, do a fresh installation of my box after the HD died and [...]
Goddamit ;_;
Taro: what is “4scrape.fcgi” path must be in lighttpd.conf? i’m try “/4scrape” but no luck, and what(where) is “api” file (maybe its fcgi)?
oh shi, i got it.. cron.sh must finish his work
and i find solution for lighttpd.conf (fastcgi.server = (“/4scrape/api”))
IT WORKS! MWAHAHAHHAHAHA!
__________________
nikki
> lighttpd.conf
Oh, whoops, I guess I should have made it available. Here’s the conf I was using.
in shared source p_4chan.py is old
[...] la entrada en el blog de Destructive Design los motivos principales para no continuar el proyecto fueron basicamente que el sitio crecio tanto [...]
Ive made a dump of all images, 181gb, ~304k files. Will get them sorted by /hr/ /w/ and /wg/ then upload to RS.
dont use RS…
yeah, what the anon said previously.
use opentracker, make a torrent and then divide each /w/, /wg/, and /hr/ image clump into 1gb 7z archives.
^ This Anon knows what he’s talking about
I am so sad! I WANT 4SCRAPE!!!!
Anon, I will be using RS ’cause my ISP limits torrents to the point it would take a year to seed 180gb. RS on the other hand I can automatically upload to at fullspeed.
Files have been categorised into /hr/ /w/ and /wg/ as per the database entries, and further divided into folders of size ~200mb each. These folders will be individually RARed and uploaded (so no split archive issues or w/eva).
So you’ll basically create ~925 200MB .rar files and upload them all to rapidshare. GL
“So you’ll basically create ~925 200MB .rar files and upload them all to rapidshare. GL”
Well I downloaded em all in under 24 hours, I dont see any other Anons delivering a torrent or anything else. -_-
Aww, I’ll miss this awesome service. :c
here is a stupid question. How do I use the lighttpd.conf? I am looking at it, and either I am too tired or my brain is fried.
> How do I use the lighttpd.conf?
You don’t — the Lighttpd web server does. It’s just a configuration file.
now how does the web server use it? yeah I am annoying, but the more I understand the more I can try to do on my own.
Thanks for all the hard work. Site will be sorely missed.
https://suigintou.desudesudesu.org/4scrape/img/356987.png
The last image I got from 4scrape ;_;
Surely there’s some money in it for whoever revives it.
Hurry and bring it back anon.
> now how does the web server use it? yeah I am annoying, but the more I understand the more I can try to do on my own.
rtfm.
Thanks for all the hard work, and sorry things didn’t work out for you. You had a great thing going.
I’ve been playing with the code on my server but the scraper keeps erroring out on finding Thread OP. I wonder if Python 3 doesn’t like the reg expressions for some reason.
> I’ve been playing with the code on my server but the scraper keeps erroring out on finding Thread OP. I wonder if Python 3 doesn’t like the reg expressions for some reason.
I doubt it’s an issue with Python (more likely moot changed the layout slightly during the last update or something) — but I was running 2.5.2 on Suigintou so there might be some weirdness.
you know what would be easier than uploading to rs and letting us spend months downloading from there?
giving us the method of which you used to download directly from the server
FUCK MY LIFE
>I’ve been playing with the code on my server but the scraper keeps erroring out on finding Thread OP. I wonder if Python 3 doesn’t like the reg expressions for some reason.
yeap, regexp is changed.. this is how it works => http://nik.bot.nu
but soon i’m shutdown this server
>yeap, regexp is changed.. this is how it works => http://nik.bot.nu
but soon i’m shutdown this server
What did you change? I thought I fixed my Thread OP problem but now it’s telling me I’m violating foreign key constraints.
Could someone pls tell me how did this Master of Walls* get all those images automatically?? Just wanna catch the concept, so I can try it myself, rather than downloading them all.
I still wanna download them all :#, but it would be nice to know the trick too =P. Please keep the server alive for a while, I’ve realized too late how to parse your .sql and extract the links =(.
*Does he have a name/nick for which we should call him?? =p
I also would like to know what host were you using?? All the hosts I know would close my account if I upload so many images.
> I also would like to know what host were you using??
Dedicated server at OVH, $200/month.
>What did you change? I thought I fixed my Thread OP problem but now it’s telling me I’m violating foreign key constraints.
added some fixes & find threads where op’s image is deleted
http://nik.bot.nu/4scrape/p_4chan.py
http://nik.bot.nu/4scrape/scrape.py
also fixed Handler.hs for new version (i’m run it in linux)
http://nik.bot.nu/4scrape/Handler.hs
omg… as they said before
“You will be missed. You provided the best wallpaper service ever. You’re efforts shall not be forgotten. Walk proud, for countless number of computers are now fashionable thanks to you.”
Thanks for everything and good luck =)
> Dedicated server at OVH, $200/month.
o.o geez..
can someone just post a link to the file list
damn it was really the best site for wallpaper
Downloading it from rapidshare will be hell unless we have premium accounts, which I do not… is there really no other way?
Can’t you code up a script real quick to grab whatever images are left on the server directly?
I’ll be limited to 24 archives a day, I don’t want to have to wait 50 days just to download this pack.
This is to the creator of 4scrape: how much would it take to get you to get this project up and running? I would gladly donate 10 or 20 dollars to the cause, and if 30 or 40 other people were willing to as well… would you consider it?
This is the previous anon, is there anyway to roll back the server to a previous date where everything was working? I mean if you removed the scraping/posting feature, would it work with little to no maintenance?
I think all of us would be happy to be able to search what has already been done, at least to collect the images we love until the server shuts down in oct.
To the buy that said he would post the batch file he made
so what ever happened to posting that batch file
>This is to the creator of 4scrape: how much would it take to get you to get this project up and running? I would gladly donate 10 or 20 dollars to the cause, and if 30 or 40 other people were willing to as well… would you consider it?
Anon #329084902384 here. I’m pretty sure there is a large number of people who offered above. With the option being available, the donations are sure to pile up steadily but effectively. I myself would even donate as soon as I get some extra cash not going into college funds. Either way though, the large number of donations from all anons here (and from /w//wg//hr/) would ensure 4scrape to keep on running.
If not, someone get the torrents up goddamnit. My desktop is like cancer ;_;
Again, this is the anon who was talking about RS uploads. Automated upload is currently in progress, I’m hoping everything will be finished in the next 48 hours or so. Also, RS has happy hours between 2am-10am CET ( http://rapidshare.com/news.html ), during which only 10% of what you download counts toward your cap. Thus on a connection of ~20mbit, with RS premium you should be able to get all the archives within maybe 4 days, if you utilize happy hours. 180gb over 4 days is pretty good by most standards.
Thanks for the great service
I have collected hr and wg images for 4 years or so manually, and I have put them on this site
http://small.calamity.org/gallery
it’s not as big as this database, but a lot of 4chan images nevertheless.
I’m current with postgres 8.3 and loved 4scrape so i’ll see if i can mangle some asp.net into makign it work again.
stay tuned to /g/ if i can.
Goddamn why wont my posts show up
RS anon here again. Here are pastes of all the RS links for the 3 image boards. The RAR files are ~200mb each, and can be extracted individually (no split archives).
paste2.org/p/362711 (/hr/)
paste2.org/p/362715 (/w/)
paste2.org/p/362716 (/wg/)
937 rar files, ~180gb, ~300k images all up.
Damn I hate wordpress, this is the 5th time I’ve tried to post these links, none of my previous posts have shown up -_-
RS Anon, (the last poster) – are those all images 4scrape used to serve?
Yes, they are.
Alright, thanks – appreciated! But why no torrent? It would have been faster that way also for people who don’t have purchased an rapidshare account.
Wow, I’m going to tak a look into these RS dls & try a few. I might actually buy premium just for this, & then dump my pass here when I’m done (don’t take it as a promise).
wow rs anon you are truly amazing, thanks for this!
Okay this with RS may not be the greatest idea, but it certainly is great, and THANK YOU x 9000. Now I might be too concerned or just plain stupid, but won’t those files uploaded to RS expire after a certain period of time? And if yes, what then? :S
http://nik.bot.nu dead already :<
The anon who uploaded that is surely making a bunch of money from it from various rapidshare actions and extras. Can’t you just distribute it via a torrent please?
A torrent would be great, but that’s like 180 GB of files…imagine having to seed all that :/
So? He could upload it to RS aswell – he could have seeded it just as well.
I suppose he’s got some reason why he didn’t do it. Maybe someone downloads it all and decides to bless us with a torrent.
He is on a network which throttles p2p. RS is http traffic which, as has been demonstrated, goes at insane speeds on his connection.
“Beggers can’t be choosers” is quite appropriate here, I believe.
Thank you rs anon.
A copy of the page should survive for posterity.
> dead already :<
not yet
Ok here goes a stupid idea: couldn’t this be solved via ftp? I’m willing to help if I can…
you should have had an apprentice trained to take over shop when you got tired this site was EPIC and im surprised no one else has tried to duplicate it… actually not surprising its tough stuff to maintain all the code,,, so thank you for wat you did give us you went above and beyond your call of duty and we all salute you for yoor efforts godspeed
how would we go about downloading these rapidshare files automatically? I’m a free user with an 8mbit connection, btw.
rs anon, you are amazing.
@preposter
JDownloader
so many desperate fans, lol
look what you’ve done! =p
almost happy hour
if you dont have premium, you would be better off waiting for a torrent
Spending countless days to sort through 300,000 images is an almost impossible feat, and it wouldn’t be very enjoyable considering 50% of the shit that comes through 4scrape is, well, shit.
I think we should take some time to compile our best finds from 4scrape, that is what it’s really about.
The best wallpapers and the best os layout/gui screenshots…
maybe someone with the images can sort through by resolution?
as long as there’s a pack with real resolutions (1920×1200+) that’d be sweet
Yup, all I want is a definitive 1920x* pack from 4scrape.
I’m still waiting for the per-board torrents.
RS Anon here again. Yes as stated RS is the only feasible way for me to distribute files that large. They will stay online at least while I retain my premium account, which will be at the very minimum, to the end of this year, and most likely into the next. Also, I don’t make any money from RS for this, only points which can be used to extend the account for an extra month.
I’m kind of tied up with work and other stuff atm, but I was also planning on doing a sort of images, certainly to filter out dupes that are just a plain resize, as well as screenshots that have taskbar and stuff showing, and then probably grouping by resolution.
I dont really have much idea about how to do alot of that programmatically, so this could be some time away.
I haven’t had time to look over any of the backend code, but I was wondering if the 4scrape creator could give a brief description of how the system finds similar images?
^ I also wonder this, perhaps something like what tineye uses…
use picasa to filter the same images, or some other third party software (only know picasa has this)
NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
>:(
:_(
RAGE!
> I haven’t had time to look over any of the backend code, but I was wondering if the 4scrape creator could give a brief description of how the system finds similar images?
It doesn’t. I was piggybacking off IQDB — to rebuild that service you’re going to have to talk to piespy on rizon#iqdb. He’s probably pissed the fuck off at me because I took down everything without telling him because I forgot (whoops! sorry piespy!)
You can always grab his source and run your own instance, if you wanted to.
[...] write a little note about it on the post here. Here’s hark’s Twitter post, and here’s a blog post explaining why he isn’t going to keep maintaining 4scrape. Anyhow, you can always check out [...]
Any news on those attempts to resurrect this wonderful project? Can I help somehow?
I NEED 4SCRAPE I THINK I’M GONNA DIE :((((
oh gay, theres no such thing as good wallpapers now..
Man, 4scrape was great. I’m gonna miss it. :c
fu, I’m not going to download 900+ rapidshare files as a free user, and I’m not goign to spend money on buying one.
someone plz upload this shit to torrents or at least filter out the crap ones and upload those but you’re not going to see me dld 300.000 images and then remove 150k of them because they are crap and another 100k because those are doubles.
the owner of 4scrape is an asshole and at least should’ve done something for us. I wouldn’t mind donating for 4scrape, but simply removing it just like that is plain evil and just retarded.
f.u.
NOOOO! sh*tstorm.
It was a long ride, R.I.P. 4scrape.
Where will i get decent wp-s now??? :(((((((
Why must every great site end up like this??? :_(
Anybody else manage to get the code to run? For every problem I seemingly fix something else pops up.
The original ran FreeBSD? Maybe I’ll load up a VM and see if that takes it without complaint.
> The original ran FreeBSD? Maybe I’ll load up a VM and see if that takes it without complaint.
i’m run this on linux within vbox
common guys, someone get a new 4scrape up and runnin, im dying here
4scrape will live.
Coming soon.
[...] 4scrape era una de las más grandes -si no la que más- bases de datos de fondos de escritorio o wallpapers de Internet. Y hago bien en hablar en pasado pues, por desgracia, el servicio desapareció hace unas semanas. [...]
>fu, I’m not going to download 900+ rapidshare files as a free user, and I’m not goign to spend money on buying one.
someone plz upload this shit to torrents or at least filter out the crap ones and upload those but you’re not going to see me dld 300.000 images and then remove 150k of them because they are crap and another 100k because those are doubles.
the owner of 4scrape is an asshole and at least should’ve done something for us. I wouldn’t mind donating for 4scrape, but simply removing it just like that is plain evil and just retarded.
f.u.
Well, with pricks like you giving him encouragement, I cant imagine why he wouldn’t want to continue.
http://nik.bot.nu/4scrape
check it out
Also, http://kotomi.doesntexist.org/
Though their hosting is shit and redirects/404s when the server is under any load.
Arooooooo…. ;___;
OMGGG GUYS THANK YOU THANK YOU THANK YOU THANK YOU THANK YOU THANK YOU SOOO MUCH :D :D
this is AWESOME. i’m gonna be there all night
GRAB IT WHILE YOU CAN
4scrape is dead wtf am i suppose to do wait my life on b now
>Also, http://kotomi.doesntexist.org/
that person seems to think 1280×1024 is a 4:3 aspect ratio.
http://kotomi.doesntexist.org/
Can’t browse past the 2nd page on any given search.
> http://kotomi.doesntexist.org/
Another sad thing is that you have to click on the 2nd page, instead of just scrolling down endlessly
Sorry to see 4scrape is down, was a lot of things I kept putting off downloading due to laziness on my part. Kicking myself for it now. Still, thanks for all the hardwork and money you put into it while it was up :) I certainly appreciated it muchly.
If anyone wants to lend a clueless anon with no knowledge of coding etc a hand I’d love to know how to get stuff straight off the server but Im guessing noone will help with that since noone wants to share the knowledge with the world and ruin their own download speeds.
Thanks for the RS Anon btw, its better than nothing and shall be my source of getting stuff unless someone helps me find a quicker way.
> Issues with Kotomi
You guys should email those to the creator, if you can find his email address (I can’t, lol).
I am planning to get 4scrape back up for good on my own site which is a private server that can handle anything. I would have to put on ads because upgrading it would cost a shitload.
I NEED TO KNOW IF EVERYONE WOULD STILL GO THERE AND TRAFFIC IT LIKE CRAZY SO THAT THE ADS WILL PAY FOR IT!
Feel free to email me to let me know things … perhaps everyone can work together on this
oh also note: it would be like 1 ad, nothing bad or in the way … also prob a donate button
I’m pretty sure you don’t have to worry about ‘traffic like crazy’…I know at least a dozen people who went there every day, so I see no reason why they shouldn’t continue doing so, if 4scrape were alive again. I totally support you and am willing to donate money if you really do this. :D
> I’m pretty sure you don’t have to worry about ‘traffic like crazy’
Umm, each month 4scrape was burning about 7TB of outgoing transfer to a quarter of a million unique IPs (~5 million pageviews/month). Quite a bit more than a couple dozen people (and that’s not even counting the hotlinkers). The software is fast as balls (since it’s a thin wrapper around the PostgreSQL database).
The first scaling issues you’re going to run into are database related — there’s too much data for PostgreSQL to effectively index (ie, you’re going to want to tie it into Lucene or Holumbus). And there’s a couple of places where there needs to be more aggressive caching (ie, thread reconstruction takes too long).
Once those are fixed, the next scaling issue is going to be your HDD read speeds — serving large amounts of highly random access is going to rape the balls out of your disk. You basically have to have separate drives for the database and image data, otherwise they’ll clobber each other’s read requests (and it’s nice to have a third drive for the system image — otherwise you’ll have really slow interactive sessions for maintaince). It’s probably more effective to not even bother with RAID and use an external backup mechanism for everything.
> Feel free to email me to let me know things … perhaps everyone can work together on this
I’ll probably send you an email later (depending on how hammered I get at the bar tonight). If by “private server” you mean a VM instance — you’re going to hit the I/O bottleneck almost instantly (due to multiplied syscall overhead from running a virtualized environment). You’re going to need a raw dedicated server to run the beast long-term.
>http://nik.bot.nu/4scrape
>check it out
wat.
[...] projektu nk do innego języka. Z pomocą przyszedł mi tutaj niedawny upadek 4scrape, bo autor opublikował źródło. Możliwość do nauki – jak dla mnie – świetna. Z jednej strony mam okazję [...]
[...] projektu nk do innego języka. Z pomocą przyszedł mi tutaj niedawny upadek 4scrape, bo autor opublikował źródło. Możliwość do nauki – jak dla mnie – świetna. Z jednej strony mam okazję [...]
When nik.bot.nu shuts down, I suggest you link or redirect to the new one for at least a day
pls tell me what you think :3
http://www.imageflash.com.ar/
I’m new to this >_>
when I get my credit card Im gonna pay a real server and upload my 4scrape collection =)
oh and don’t worry about thumbs’s size (i was experimenting), from now on I will stick to the traditional 300x
@ImageFlash:
You need some better thumbnails and to load fewer of them at once. Also categories are silly.
:<
ok thx
but these are good ones, right?
http://www.imageflash.com.ar/wallpapers/0/Anime/all/all/mayor/id/desc/Go!
i just need a better server so i can upload my walls massivelly.. if i upload more than 10 or 20 in a row with this 000webcrap it disconnects me >_>
when does the newe 4scrape ends?…i mean mercury lampe
Update from RS anon:
If you are downloading my RS links, dont bother with 184-236 from /w/ or 280-370 from /wg/.
Looking over the images it appears the server choked on quite a few (read: 30gb) and all I got was files full of null bytes.
I dont know what exactly the problem is, the files don’t 404 when requested (so I assume the name is valid), they just seem to do nothing/send blank data.
Image numbers from what I can tell are ~ 305xxx.jpg-356xxx.jpg (pngs affected too).
Example URL: http://suigintou.desudesudesu.org/4scrape/img/356495.jpg
(if the admin wants to take a look).
@RS Anon August 12th, 2009 7:14 am
Perhaps 4chan changed something but scraping script didn’t got updated and kept downloading void.
forgot to ask: How do u sort those images into separate boards? Can’t figure it out on that database structure.
> Image numbers from what I can tell are ~ 305xxx.jpg-356xxx.jpg (pngs affected too).
Dunno, it’s like that on the HDD too. Something must have shat itself along the way.
> How do u sort those images into separate boards? Can’t figure it out on that database structure.
SELECT img_path
FROM s_images i
NATURAL JOIN s_posts p
NATURAL JOIN s_boards b
WHERE b.board_sname = ‘w’ — or ‘wg’ or ‘hr’ whichever you want
;
That’s a slow as hell query, so there’s a couple of tables where it’s cached (ie, PostgreSQL doesn’t really support materialized views so I hacked it with some extra tables). You can just do a `SELECT img_path` from the `v_random_X` tables where X is the appropriate board_id.
etc.
@Anonymous August 12th, 2009 8:15 am:
Thanks for that.
I did a bit of “benchmarking” as I’m yet unsure what database to use for storing all this data, and those are some of my results:
Query was like that: SELECT i.img_path FROM s_images AS i INNER JOIN s_posts AS p USING(img_id) WHERE p.board_id = 1;
MySQL:
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.0669901371 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.83250784874 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 6.55538511276 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 3.99688696861 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 3.97126293182 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.05414795876 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.01427602768 seconds
PostrgreSQL:
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 3.97423791885 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.28063106537 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.31740093231 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 2.77022504807 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 5.42018318176 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 4.39519619942 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 2.44304203987 seconds
Query was like that: SELECT i.img_path FROM s_images AS i INNER JOIN s_posts AS p USING(img_id) WHERE p.board_id = 1 LIMIT 5000;
MySQL:
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.0116081237793 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.010880947113 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.0112218856812 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.011125087738 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.0112619400024 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.0116970539093 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.0110239982605 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.0111429691315 seconds
PostgreSQL:
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.889919042587 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 1.06394696236 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.896944046021 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 1.0629401207 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.879417896271 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.93269109726 seconds
paziek@lolita:~/www/deny$ php phpinfo.php
Query took 0.906115055084 seconds
Difference is so huge when you added LIMIT to the query, that I’m wondering if I’m not doing something wrong.
PostgreSQL has clean dump downloaded from here, while MySQL has only those 2 tables that are used with a bit different (MySQLish structure). On MySQL part I have only primary indexes on img_id (s_images) and post_id (s_posts).
I was using this script (bad PHP, I know!)
http://91.121.39.10/phpinfo.php.txt
Almost forgot:
paziek@lolita:~/www/deny$ mysql –version
mysql Ver 14.12 Distrib 5.0.67, for debian-linux-gnu (i486) using readline 5.2
paziek@lolita:~/www/deny$ psql –version
psql (PostgreSQL) 8.3.7
contains support for command-line editing
Your benchmarks are almost entirely I/O bound. Consider doing some more complicated queries (ie, given a thread and board ID fetch and order all the posts from that thread) instead of just grabbing lots of rows. Try to make the benchmarks mimic what your application is going to be doing.
Also note that you’re probably using MySQL’s MyIASM backend which cuts a lot of corners to be fast — including ignoring ACID compliance.
Finally, if you do decide to run MySQL you’ll have to rewrite at least a few of the queries in the backend code which use PostgreSQL-only features (specifically, the PostgreSQL fulltext indexing support). MySQL also supports fulltext indexes but uses a completely different syntax.
Why not hire a few people from here that are decent with the code, teach them the ways, create a donation option and a membership option with a few mindless perks(that people from 4chan will eventually get around to get additional content for free) and just go from there? I mean… all these people are right… your site was the light in the dim dim world that is our lives, as sad as it is, I looked forward to seeing new wallpapers every day and you are the ONLY person who was willing for so long to keep it up. So whether or not you end up continuing the site, I thank you for your years of faithful and unrelenting service to us… your followers.
> Why not hire a few people from here that are decent with the code
1) I don’t make enough money to pay other people to maintain my site.
2) I don’t want to spend more time on this project — it was designed to be a maintenance-free toy and has, over the past year, become a chore to keep running.
3) As far as I’m concerned, the entire codebase needs a significant overhaul (ie, complete rewrite) to fix the problems listed above.
The best case scenario for 4scrape’s resurrection is for a talented person to come along, pick up my data and independently write their own improved implementation of both the back and front ends (please note that you’ll need to integrate a real fulltext search index).
Unfortunately I don’t see that happening. I expect at least one of a couple groups picking it up, as-is, and running it well for about a year at which point it’ll buckle in on itself again.
And that’s fine. Nothing is meant to last forever.
@Anonymous August 12th, 2009 11:33 am:
I did that and MySQL was still much faster. But what I figured out is that MySQL was actually caching those queries and if I used “brand new” thread_id then it was more or less like PostgreSQL (postgre a bit faster).
Odd thing is that using ways described in MySQL manual to disable cache did not have effect. Perhaps I would have to compile it from source with explicit flag disabling cache.
I did some experimenting with Postgre cache (increased shared mem and cache size) but it improved times only a bit (around 25%, where MySQL has 100x better times for cached query). Perhaps I’m doing it wrong and I did not enable caching. Not sure, I guess I should go – for now – with MySQL after all if I don’t know how to “fix” Postgre.
I would like to ask you something: Why are you putting so much pressure on fulltext indexing? From what I understand all it does is let you order results by relevance to the search term as well as enables a bit more complicated search terms. Does it speedup query time?
While it is good to hava results sorted by that in search engines like google or for example in blog/newspaper/etc I see no reason to do that with posts from 4chan.
> MySQL was actually caching those queries and if I used “brand new” thread_id then it was more or less like PostgreSQL (postgre a bit faster).
PostgreSQL *should* cache the queries too. I always found that the default memory limits are waaaay too low to do it effectively though — on a production machine you’re going to want to dedicate at least 2GB to it (and you’ll probably have to tweak some kernel settings to allocate that much).
The tuning guide says that’s effectively a moot point if you’re on Windows, though.
Also note that PostgreSQL’s effective_cache_size is for the disk cache, not the memory cache. The memory cache is all taken from shared_buffers.
Another note is to make sure if you’re doing any queries with both an ORDER BY and LIMIT clause to make sure the entire result set (before the LIMIT) will fit in the work_mem space (which is per-connection, so it doesn’t need to be that high). If it doesn’t, PostgreSQL won’t be able to sort in-memory and that’s slow as balls.
Granted, if you’re sorting on an index used by a WHERE clause (or possibly a JOIN constraint) it should be free (check EXPLAIN to see if my guess is right >_>).
I definitely think the threads should be cached outside the database entirely — I was going to set it up so that the FCGI backend writes out raw HTML/JSON which gets served up (and handles a 404 thread by writing out the file and then 301′ing). Then have the scraper delete/refetch the thread to regenerate the cache automatically.
No reason to hit the database when (the majority of) the thread contents don’t change.
> I would like to ask you something: Why are you putting so much pressure on fulltext indexing? From what I understand all it does is let you order results by relevance to the search term as well as enables a bit more complicated search terms. Does it speedup query time?
It depends on the indexer, but yes yes and yes. PostgreSQL and MySQL aren’t designed to be fulltext search engines — though they have functionality which is okay for the most part it’s ridiculously slow and really doesn’t compare to using a piece of software designed explicitly for the job (ie, isn’t designed to query across multiple fields of half a million rows).
There are also some weird performance issues with PostgreSQL combining a fulltext query with an ORDER BY clause. The 4scrape code hacks around it by ordering the posts in-memory by post ID (which are guaranteed to be ascending by time because they’re assigned with a linear sequence).
A real fulltext search engine could allow you to order by either the post ID (as 4scrape currently does) or relevance (which AFAIK PostgreSQL has shit support for currently — though MySQL’s isn’t as bad).
Using a real indexer like Lucene of Holombus (though I haven’t used Holombus yet) would make the search queries (ie, the fulltext ones) almost instantaneous instead of being a slow crawl. The non-fulltext queries (ie, resolution and friends) are handled perfectly fine by an RDBMS.
Anyway.
I think the thing that pissed me off most about MySQL was needing a startup script to REPAIR all the tables automatically for when something got restarted violently. A couple of times back when 4scrape was running on MySQL (waaaaay back when it was still written on Python running through Apache’s mod_python — I have that code too, if anyone’s interested. Spoiler: it will rape the balls out of your machine) 4scrape went down for a couple days a couple times because I didn’t notice the MySQL database decided to shit bricks. PostgreSQL is a solid rock.
4scrape is gone?
DAMNIT WHY HAVE YOU FORASKEN ME INTERNET?
I’ve been working on a revival project too, but I’m not using any of the existing codebase. I’m using php, mysql and jquery and I would say that it’s currently about 65%-70% complete. Got a working scraper, the database is pretty much sorted, got thumbnailing, ‘infinite’ scrolling etc done. My major concerns are probably related to the scaling and bandwidth issues. I don’t really have any experience running bandwidth intensive sites so I may need some help eventually with regards to hosting options. I’ll be posting back in a few days with a beta before looking for a host.
> I would like to ask you something: Why are you putting so much pressure on fulltext indexing? From what I understand all it does is let you order results by relevance to the search term as well as enables a bit more complicated search terms. Does it speedup query time?
While it is good to hava results sorted by that in search engines like google or for example in blog/newspaper/etc I see no reason to do that with posts from 4chan.
true, i would be fine with just the “random” feature, and watch unlimited wallpapers (works) =)
RIP 4scrape. Thanks for all the effort you put into the site.
sum1 injected code in my comments section..
very funny ¬¬
> I’ve been working on a revival project too, but I’m not using any of the existing codebase. I’m using php, mysql
Make sure you throw an opcode cache onto your PHP stack. That’s probably the most overlooked optimization ever and is really easy to do.
Scaling PHP is really easy too — you just lease more application servers and hook them up behind a reverse proxy. Easy, but expensive >__>
As far as hosts go, I can recommend OVH (and to a lesser degree, FDC). To lease from OVH you have to be a member of the EU or know someone who is, but their bandwidth is cheap as hell. FDC has cheaper, better hardware, but more expensive bandwidth, shit customer support and has to deal with bullshit US laws.
4scrape was hosted at OVH.
@ImageFlash
The “FREE WALLPAPERS!!!” need to be removed. It’s just pointing out the obvious
@ImageFlash:
quote test (original post is above)
Dude, i know its obvious, but some people come to my site without knowing what it is about and can’t figure out by themselves.. Some people doesn’t even know what a wallpaper is =S (hard to believe for wallpaper freaks like us =p)
Trust me, i didn’t like to put that, but i had to
Besides, it also helps with on-page search engine optimization.
Thx anyway =)
Looks like someone ass-raped Kotomi?
That site was shaping up to be an excellent alternative. Stupid motherfuckers…
btw it’s almost done
[moo@vbox ~]# fs
Free space on nik.bot.nu
/dev/sda6 63G 33G 31G 52% /
Can i ask what happened with 4scrape? i just know about this web site, its a wallpaper site? its over? . . .
>Can i ask what happened with 4scrape?
it was hard to mantain
>i just know about this web site, its a wallpaper site?
the best wallpaper site ever
>its over? . . .
we’ll keep it alive.. in our hearts :3
So nik.bot.nu it sgoing to be down in any moment and that`s it?
I can belive that anyone can make again 4scrape…
Do you relize that it was the only wallpaper site with many wallpapers of any size? For example is too dificult get a wallpaper site with 1920*1080 wallpapers… for example.
> Looks like someone ass-raped Kotomi?
pfft, Kotomi had shit hosting. 4scrape was handling more than three times that much traffic.
> I can belive that anyone can make again 4scrape…
Anyone can. Hell, you can if you want to — I’m sure there are plenty of people running local instances on their boxes. It takes a damned fool to not only make an instance but to make it publicly available for people to rape freely.
inb4 someone publishes a local installer bundle and zip/orz get raped to their knees.
>It takes a damned fool to not only make an instance but to make it publicly available for people to rape freely.
Thats me :D
Patience please everyone its a big project.
4scrape is a MUST on internet…
> 4scrape is a MUST on internet…
Please do us the honor of installing an instance and paying the time and money it costs to keep it running. Thanks!
Love,
The Internet.
I’d like to throw my hat into the ring to reimplement 4scrape as well. I’ve got experience in scaling Python using NoSQL databases.
I haven’t read through the whole thread, but I saw a few other offers for help. My question is: are any of those promising or are they still just words? Should I even bother starting now?
(I should also note that I have money to throw at dedicated servers, probably unlike some of the other coders)
@Zain: Go ahead. I doubt they have money for a dedi that would hold traffic that was going throu 4scrape. Even if they do make some front and backend – it will die by sheer volume of visitors. And since you claim to have money to throw at dedicated servers, not just one – hell yeah :)
I did wanted to mirror it as well, but I think there is a few already going on, so I’ll wait and see how it will come out.
Thank you Hark for your great effort. 4scrape was one of the best sites of whole intarwebz.
Another site with wallpapers in 1920×1080? X_X,a good site like 4scrape not the tipical sites that don’t have nothing…
I have http://nik.bot.nu/4scrape/ but we know that is for some time and then bye…
When http://nik.bot.nu/4scrape/ dies, is it going to come back? Or does he mean by “temporary server” that it’s not coming back
4walled.org
New scraper. Only 10k images though.
> When http://nik.bot.nu/4scrape/ dies, is it going to come back? > Or does he mean by “temporary server” that it’s not coming back
at first i just wanted to try to run the server only for myself (for selftraining), i didnt plan that it will work constantly =)
i have installed the virtual box (set hdd = 60gb) on my personal computer with windows_xp and for two weeks the outbound traffic already is 2.5TB O__o
I think my provider will block me for it :)))
Welp, I’m willing to donate money or help out somewhat for another 4scrape website/server alike. I loved the website, it provided me with a lot of backgrounds. All I’m waiting for, is that image data to be placed on bit-torrent. If anyone wants to try to restore 4scrape or build a clone exactly alike. Contact me at Sakurazaki.toybox@yahoo.com
@Nik, at your 4scrape dump we can see new posts day by day, how can I post something or browse THREADS list? Thanks.
THE RAPIDSHARE LINKS ON PASTE2.ORG ARE GONE CUZ THE SITE IS GONE!!!!! COULD SOMEONE REPOST LINKS PLEASE
You’ll have to speak LOUDER, I think on one heard you.
Now to go back on topic. I hope that once someone brings up a permantent (as in: ‘non-temporary’) 4scrape, that ‘someone’ can also bring the old pics back. That makes the random button waaay more exciting ;P
In response to “RS Anon August 12th, 2009 7:14 am”, is there any news on the missing images?
>In response to “RS Anon August 12th, 2009 7:14 am”, is there any news on the missing images?
>> Image numbers from what I can tell are ~ 305xxx.jpg-356xxx.jpg (pngs affected too).
>Dunno, it’s like that on the HDD too. Something must have shat itself along the way.
^^ this. Obviously there is no way of getting them seeing as they weren’t really scraped in the first place.
> THE RAPIDSHARE LINKS ON PASTE2.ORG ARE GONE CUZ THE SITE IS GONE!!!!!
^^ WTH are you on about? paste2.org is still up, and RS links are still live:
paste2.org/p/362711 (/hr/)
paste2.org/p/362715 (/w/)
paste2.org/p/362716 (/wg/)
NOTE: dont bother with 184-236 from /w/ or 280-370 from /wg/, the images in those archives are corrupt (see previous post), and hence I removed them RS.
793 rar files, ~150gb, ~250k images all up.
When i cbf I’ll make /wg/ packs of images of diff resolutions.
@SB, u cant post, only view images and threads
to view thread open any image, click on “show posts” and then on yellow link “/x/xxxxxx”
>I hope that once someone brings up a permantent (as in: ‘non-temporary’) 4scrape
i hope too
4walled.org has potential =)
Nik.bot too, but well we all know that what is his future =(
Ty. Ive added random and other features at the request of /wg/
the database is small but its growing. 15k images
4scrape will never be forgotten
NOOOO!
Internet will never be the same without 4scrape.
We all hope to see it sometime again!
is anyone still sorting and making a torrent
Anyone who got all the 150gb of images that can make a torrent? Rapidshare sucks.
Acid Burn, if you are still reading here can you e-mail me at:
jmetcalf1177@gmail.com
I have some questions for you about the 9scrape source.
Sorry I’ve been busy with different things. I’ve separated my project from 9scrape. Current project can be found at http://scr4pe.bot.nu
I am redoing the databases and I still have several things to work on. At present, normal mode, random mode, scraper and infinite scrolling are some of the features that are mostly complete. I’ll spend a bit of time on all the searching features, user system and a view page with image info (dimensions, size, rating, views etc). As I’ve said I’ve been busy with other things so I’ve barely worked on it in the past week but I’ll try and get something done over the coming week. Note that it is not yet on a production server per say but feel free to rape the 500GB cap. Also, it will not work in IE but should display in all other decent browsers. I’ve gotta read more about html5 or drop it altogether.
Have you put any thought into scalability?
there is a problem with sc4pe.bot.nu
I hit the bottom
sc4pe.bot.nu good proyect too, but you need to put more things to the site and more resolutions like HDVTV resolutions…
But is a good alternative ^^
really sad to hear of 4scrape’s demise. thankyou for the trouble you took and for the wonderful resource you provided, i got some wonderful wallpapers from it! good luck in the future!
once you’ve 4scraped, there’s no going back :(
NO FAUKING WAI! NO NONONONONONONONONONONONONONONONONONONONONONONONONONONONONO
4SCRAPE IS GONE?!
SEND THE TORRENTS NAO!
HOW_FAR_CAN_I_GO??>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>LOL
So like, have we settled on what site is going to replace 4scrape yet? Or are there still like 11ty different prototypes still being worked on?
[...] na stronę 4scrape. Niestety już nie istnieje (Oficjalne info o przyczynach zamknięcia na tej stronie.), strona przerosła możliwości twórcy. Nie ma szans na [...]
@Acidburn: great!
ps: nik.bot.nu disk 83% in use, after that i turn off the virtual box.. sorry.
Fuck the guy who put over 700 archives on RS. That isn’t any more feasible a solution than sharing 150GB through a torrent.
nik: thanks a lot for what you’ve done.
http://nik.bot.nu/ is what all the guys ‘working to bring 4scrape back up’ should aim for. No sketchy gayish stuff, and more importantly: NO HITTING THE BOTTOM!
Is there not a way to get ftp accsess to the 4scrape server? i know this would rape the server, but if you limit how many connections that can be connectet at the same time you might be able to handel it. if not we’ll just have to wait for the mega torrent of all torrents to be released.
If such a torrent is released i will use my company server to seed it! God speed people
> ftp access
No. Listing a directory with 300,000 files in it would probably crash your FTP client.
As a note, HTTP access to the image data is either coming down soon (depending on what the billing schedule was, I can’t remember).
GODOMMIT. ;-;
>Fuck the guy who put over 700 archives on RS. That isn’t any more feasible a solution than sharing 150GB through a torrent.
So shut up unless you have a better idea. If you guys are really so desperate for a torrent, get a copy of the DB and dump the image links yourself, then download them all over http (its not that difficult, esp if your internet connection doesnt have a monthly cap)(actually the images might be offline by now). Or download the images from RS and make a torrent. I have already explained why I chose RS over torrents, so far it doesnt seem anyone else has bothered to download and share the entire set or images, so stop ur goddamned bitching, either take it or leave it.
Dude, can’t get to your lighttp config file.
REPOST pretty please :D
Server died, can no longer grab images using script :’(
Ive downloaded almost all of /w/ and /wg/ .rar files from RS with no problem..
Just gotta know the right tool, lol
Check this out http://rdesc.com/
Enjoy
>If you guys are really so desperate for a torrent
It’s not that we’re desperate, it’s that we’re willing to do it but we don’t want to put up with any bullshit along the way.
I’m not desperate for a torrent, im desperate for 4scrape to continue….
I just want a helping hand to get it going again, start it fresh, on an empty database, within a year a large amount of the content will be back anyway…
Please post the lighttpd.conf file again to help with this…
@solaris http://suigintou.desudesudesu.org/4scrape/lighttpd.conf
ps. i’ve just downloaded all ~304k files from 4scrape and i have ~55k on my server, i’ll try to create a torrent (i will split each archive on 10-15k files)
I need a torrent of all the 4scrape wallpapers
thepiratebay failed to uplaod a torrent…
here the link http://nik.bot.nu/4scrape/4scrape.torrent
btw torrent size 214GB
can u make each rar smaller
fat32 does not support this
sorry i cant, i haven’t free space on disk.. try to use ntfs/ext/jfs :\
maybe someone will make megaupload mirror
will be perfect
214GB TORRENT
http://nik.bot.nu/4scrape/
Will seed for life. (if I can dl it all)
Bringing the torrent down now, will seed for life too if I can grab it all….
Estimated time remaining over 3 weeks, but got a good ratio going :D
can someone upload a mirror to megaupload or rapidshare
or make a different torrent for peoplelike the guy with fat32
many ppl have this problem
rapidshare already was uploaded by RS anon
> The RAR files are ~200mb each, and can be extracted individually (no split archives).
>
> paste2.org/p/362711 (/hr/)
> paste2.org/p/362715 (/w/)
> paste2.org/p/362716 (/wg/)
>
> 937 rar files, ~180gb, ~300k images all up.
nik you should buy expensive web hosting, people would definitely donate to see a 4scrape come back alive.
rapidshare already was uploaded by RS anon
> The RAR files are ~200mb each, and can be extracted individually (no split archives).
>
> paste2.org/p/362711 (/hr/)
> paste2.org/p/362715 (/w/)
> paste2.org/p/362716 (/wg/)
>
> 937 rar files, ~180gb, ~300k images all up.
yes but yours is 214 GB while his is onle 180 GB
any difference? u have much more
The difference (this is just me guessing) is that he included all the images that were gathered in his 4scrape during the 25 days it was up.
4Scrape has died. This is truly a terrible loss to the internet, for it was easily the greatest wallpaper site ever.
Posting
How big should the connection // server be to run a 4scrape?
I am maybe willing to do it, if i just knew how much capacity i needed.
Hi there. I am on my way to set up a new site with wallpapers. Please feel free to suggest what you would like see on this new see. I’ll try my best to bring up 4scrape archive online. Contact my via twitter at http://twitter.com/I__o__I
I’m working on downloading everything so I can make a megaupload mirror.
Also, my server provider is dropping my 2.5 TB limit shortly (to unlimited bandwidth), so I can rehost stuff.
>can u make each rar smaller
>
>fat32 does not support this
Fat32 supports 200MB files, unless you’re pulling from something else.
Uploading at a maximum of 120kb/s to 4 users at a time, will continue to seed once downloaded (in 3 weeks)
damn, i will need to get a new external hdd
after that i will also seed for life!
You shall never be forgotten
Good night, sweet prince.
Firefox can’t establish a connection to the server at suigintou.desudesudesu.org.
Sad error I get now. Is the torrent and rs the only way to get the image anymore?
A legendary website, man, thanks and hopefully you bring it back one day if everything falls together.
http://nik.bot.nu/4scrape/ There is the torrent, made me laugh at how big it is xD
http://4walled.org/
http://217.69.173.118/~hakthepl/projects/scr4pe/index.php
To the creators of these two scrapers.
THANKS :3
Moar wallpapers \m/
i am disappoint. i will revive this myself with your source code. it will be better. i will call it over9000scrape and then you will be all hella jealous and i’ll be all “i’m so goud at it” and then you’ll see.
Just in case someone cares =p,
last hour changelog :
-removed advertising
-added random feature
http://www.imageflash.com.ar
-infinite scrolling is on its way
my main problem would be the hosting, gotta get a job xD or donations..
you can leave your suggestions in the comments section if you want
cya
Was such a great website… Too bad :(
Noooo !!!
I’ve been checking the site every few days since it went down for updates; guess I was just kidding myself, but hey, 4scrape was amazingly useful, so thanks in bucketfulls for providing the service while you could. It was truly enjoyable fiddling through a cavalcade of amazing and not-so-amazing images. R.I.P.
So sad. I’ll never take it off my favorites.
Thankyou so much.
anyone have /wg/ in a torrent you would be my hero
i just ate a mosquieto. thought you would want to know.
No flashy wank.
Sooo sad… :( I find the service bookmark it download a crapload of stuff… stop for a couple of weeks and now its Gone :(
SAD SAD SAD… anything even slightly similar ?
Thanks. and Hope you come back 4scrape
:)
In honor of 4scrape: http://4walled.org/
Same thing, new coder, new code.
And smaller thumbnails <.<