Any way to user defined FTP search engines?

[otrov]

Hi,

FTP search plugin is great, but it doesn't work anymore unfortunately.
You just provide 2 engines, and as time goes by, those aren't working anymore, and there is no option for user intervention.
By intervention, I mean the same as you can do in any browser search bar, by defining your search engine and providing parameters.

Here is what AS, does:

1. Uses Host: http://www.filesearching.com, and gets response:

HTTP/1.1 403 Forbidden

2. Uses so called Archie (Host: ftpsearch.archive.sunet.se:8000, Host: ftpsearch.elmundo.es:8000), and gets timeout:

HTTP/1.1 502 Connection Failed

Am I only one using this service?

[otrov]

I opened both libraries (from ftpsearch plugin, parser folder)

1. Archie
None of provided (commented) Archie hosts works (from 1253 rcdata resorce)
I'll check later if I can find working one with Google, and provide working plugin to others.

2. Filesearch
Problem with filesearch engine can be solved by adding Referer header to request, i.e.:

Code: Select all

Referer: http://www.filesearching.com

This is my deduction of regex scheme for filesearch engine, thou I doubt anyone will find it useful:

Code: Select all

RCData resource: regex pattern
------------------------------------------------------------------------------------------------------
1150: ^ *?(\d+?) *?<img src=/img/(.+?)\.gif width=16 height=16> (?:<b> *?(\d*?)</b>| *?<(.*)?>) ...
           ^--^                   ^-^                            ^----------------^ ^--------------^  ...
            |                      |                                       |                |         ...
           idx                 icon name                               file size           tag        ...


 ...  (\d\d)/(\d\d)/(\d\d\d\d) <a href=.*? class=ls>(.+?)</a>(?:<a href=.*? class=lg>(.*?)/?</a>      ...
 ...   ^--------------------^                        ^-^                              ^-^             ...
 ...              |                                   |                                |              ...
 ...  obvious date (not present)                  host name                        path name          ...
      
 ...  <a href=.*? class=lf>(.+?)</a>|<a href=.*? class=lg>(.*?)/?<b>(.+?)</b>/?</a>)$
 ...                        ^-^                            ^-^       ^-^             
 ...                         |                              |         |            
                          file name                      path name file name          

------------------------------------------------------------------------------------------------------
1151: <a href=(.+?)><img src=/img/moreengl\.gif width=\d* height=\d* border=0></a>
               ^-^
                |
       link to next page
------------------------------------------------------------------------------------------------------
1152: ^ *?(\d+?) *?<a href=/cgi-bin/s\?q=.*?&l=en&e=on>(.+?)</a>$
  ^   
  |
not clear as I can't find similar match
------------------------------------------------------------------------------------------------------

Which again doesn't work, as any host provider changes will affect this fragile scheme, as it does.

Things like this should be provided to end user in open format. Like XML. Closing it is always bad idea.
Now plugin can't be easily extended and more importantly issues can't be repaired (until the next version, and next minor change or third parties end)

[Ether]

The developers abandoned this plugin.

[otrov]

Too bad, it integrates just fine with FTP plugin

In the meantime I found one working Archie server:

http://archie.icm.edu.pl/archie_eng.cgi

but unfortunately in does not accept GET request, but works just in POST

It would be great if anyone replies with working Archie server, or help in changing request type from GET to POST, or help in changing request type by appending additional header so that filesearch module, returns result. I can correct regex patterns myself.

[otrov]

As I couldn't find working Archie server with GET method, I focused on filesearching module, and found almost identical service: http://www.freewareweb.com/ftpsearch.shtml

Server uses ftpsearch.pl script, and from the contents of search results and parameters it seems like it's same service as filesearching. I patched AS module, and additionally changed regex pattern to match new scenario. I changed resource 1150 and 1151, but I guess 1150 is most important:

Code: Select all

()()&nbsp; +(?:<b> *?(\d*?)</b>| *?<(.*)?>) (\d\d)/(\d\d)/(\d\d\d\d) (?:<a href=.*? class=ls>(.+?)/?</a><a href=.*? class=lf>(.+?)</a>|<a href=.*? class=lg>(.*?)/?<b>(.+?)</b>/?</a>)

I monitored AS with modified plugin, and it receives correct data and regex pattern is tested to return correct match groups, but unfortunately that's not enough for valid results. If someone can look into it, please do. I attached the parser.

Also it would be nice if plugin is not supported anymore, source code to be released, mainly because it's not working or course.
It is mentioned in Ether's link that it's obsolete, but it doesn't look like there is a replacement to make it obsolete