|The Game||Star Fleet||Other||Forum|
« About the Site / Search
The site search is based on Sphider: originally written by Ando Saabas and kept updated and expanded by captquirk at World Space Flight.
In order to index changes to the site automatically, the indexing spider is triggered at the top of every hour via a cron job that executes a bash script. This script, in turn, passes the scan variables to the spider PHP script and executes it.
There may be a slight lag, therefore, before the search database reflects the newest changes to the site.
The hourly schedule was chosen to keep the search results as current as possible while reducing workload on the file and application servers.
Site Database Querying
I kept the two boxes on the search page that allow you to run a query against the site's own news/content database: either in the titles or the content. These filters match the entire string that you enter, rather than actually searching, where it goes after multiple, individual terms. This can be useful though, if you know what you're searching for, and there's no indexing lag, since it's querying the existing site database directly.
The SFXB forum is excluded from Sphider's search results since the PHPBB already has its own search facility.
Testing of the indexing process revealed that local indexing scripts usually can't access pages with spaces in their URLs. The server throws a 301 soft error that redirects a browser to the 'correct' URL with the spaces encoded as "%20", but often not other user agents.
So, the site was re-coded to make sure that it always 'pre-encodes' spaces in URLs as "%20", and does similar for any other special characters, appearing in URLs, that caused issues with indexing. This eliminates any subtly different URLs for the same page being presented across the site that the search engine would to have to deal with as duplicates. Also, the fact that the server isn't having to redirect the user agent, and serve two pages for every request, when it sees spaces in the URLs, now means that SLIR runs a lot faster on media pages with lots of images. That led to a noticeable performance improvement.
The use of querystrings was cut back, again to reduce the number of URLs, in the index, that display similar content. For example, the media subsystem, where you can choose which amount of detail to display about images ... The alternative was to use sessions to pass the data between page loads and this made it neater and easier to make this selection persistent as the user navigates the site. Similar improvements were applied to the news subsystem.
The site's error-handling system has also been expanded and improved, so that error pages including those handled by the site, rather than the webserver present a "noindex" meta tag in the HTML header, to prevent sphider from indexing them.
Lastly, the expanded main menu was able to be cleaned up, with much of its functionality moved to the search page: reachable via the search box on the main menu.
Lastly, I used the 'preserve details' englargement method in Photoshop to generate a large banner for the search area of the website, showing Wingblade landed on an alien shoreline, on the sanctuary planet that features in level 9 as well as Rain Song.
2021.04.05 - With version 10 of the site, site search was handled by passing queries to the user privacy-centric search engine: Qwant. the "Contact" link on the main menu was replaced with "Search", which expanded a dropdown menu to allow the user to search the site contents. There is the option to query titles and contents of the site's news database and a link to the SFXB forum's search page.
I had a number of problems with relying on an external search engine, ranging from the results being outdated or incomplete, to the inability to make the results match the appearance of the site.
2022.06.23 - So, I went hunting for a local search engine script, similar to the Fluid Dynamics engine at Ex Astris Scientia.
Directory contents last updated: 2022.07.24
TV Shows - Dan Dare
This site and content is unofficial; © 2002-2023 Piers Bell and other authors.
Star Fleet, X-Bomber et al. © Enoki Films.