using scrapy with solr

Feed

17 Jul 2025

Five year anniversary

17 Jul 2024

Four year retrospective

16 Dec 2023

Milestone: 2,000th site indexed

19 Aug 2023

Introducing the redesign and first major open source contributor

17 Jul 2023

Three year retrospective

10 Dec 2022

Migrating off AWS has reduced running costs by 80%

29 Oct 2022

Lots of new web feed (RSS and Atom) related functionality

18 Sep 2022

New site listing workflow and search as a service improvements

12 Jun 2022

An update on the automated SEO searches issue

14 May 2022

Almost all searches on my independent search engine are now from SEO spam bots

12 Mar 2022

Milestone: 1,000th site indexed

5 Mar 2022

Switching to cheaper hosting and stopping indexing wikipedia to cut costs

8 Jan 2022

searchmysite.net retrospective and future plans

30 Oct 2021

searchmysite.net now with added Wikipedia goodness

23 Oct 2021

Some of the challenges of building an internet search

11 Jun 2021

Progress update Q1 & Q2 2021

30 Jan 2021

searchmysite.net: The delicate matter of the bill

17 Dec 2020

Advertising and search engines: When it is okay to mix and when it is not

12 Dec 2020

searchmysite.net is now open source

5 Dec 2020

Relevancy tuning for searchmysite.net

29 Nov 2020

Web analytics on searchmysite.net

22 Nov 2020

Improving stability and fixing issues on searchmysite.net

21 Nov 2020

searchmysite.net now has its own blog

9 Oct 2020

Adding a simple search page to my personal website with searchmysite.net

25 Sep 2020

searchmysite.net update: Seeding and scaling

18 Jul 2020

searchmysite.net: Building a simple search for non-commercial websites