The crawler has been running pretty well with only minor tweaks from day to day which sometimes show up as blips in the graph. It was also down for a few days due to a failing hard drive.
Yesterday the crawler got into the Foxy network again which uses the same protocol as G2 but is a private network separate from G2 probably using GnucDNA’s authentication scheme. Their network is significantly larger than the G2 network as can be seen from the spike history graph. Since it is so much larger it presents problems for crawling and since it doesn’t represent the true open network of G2 I have filtered it out from being crawled.
henryhenry | 23-Jul-08 at 12:03 pm | Permalink
I stumbled on this site looking for original Gnutella crawler. Nice work btw. Can someone explain me what is Foxy network? My guess goes to something either Chinese or in Israel.
Also does anyone still have the source to the original gnutella crawler? Or know if it is able to handle the current gnutella network size? My homebrew crawler runs till infinity.
Sorry for slightly OT. This seemed the sort of place somebody might know.