SearchAPI < Local

You are here: Foswiki>Local Web>ImcOceania>ImcOceaniaConf2004>ImcOceania2004Workshops>CommonAPI>SearchAPI (30 Sep 2005, AlsteR)Edit Attach

A Common search API

If we wish to search for "frodo" on oceania indy sites, the URL differs from site to site:

Perth, manila (activismo):

http://perth.indymedia.org/index.php?PHPSESSID=a0965b6536b0d7bc6f24ef11637637f9&action=search&words=frodo

http://perth.indymedia.org/?action=search&words=frodo

adelaide, brisbane*, jakarta (active):

http://adelaide.indymedia.org/search-process.php3?medium=all&keyword=frodo

[*brisbane has a problem with zero results:

http://brisbane.indymedia.org/search-process.php3?medium=all&keyword=frodo

Warning: Division by zero in /var/www/www.cat.org.au/active/brisbaneimc/webcast/search-process.php3 on line 185]

melbourne, qc (sf active):

http://www.melbourne.indymedia.org/news/?comments=yes&medium=text&keyword=frodo

http://qc.indymedia.org/news/?comments=yes&medium=text&keyword=frodo

sydney (active):

http://sydney.indymedia.org/search-process.php3

aotearoa (dada):

http://indymedia.org.nz/mod/search/dosearch/index.php?dosearch=1&searchtext=frodo

Mir:

http://sandiego.indymedia.org/sandiego/servlet/OpenMir?do=search&search_content=frodo&search_submit=Search%21&search_boolean=or&search_creator=&search_topic=&search_hasImages=n&search_hasAudio=n&search_hasVideo=n&search_sort=score

These differences make it more difficult to have a common, distributed search (triggered, perhaps, from a form on oceanea).

A Common URL

We could settle on a URL of the form

city.indymedia.org/search?query=XXXX&maximumRecords=NN&startRecord=JJJ&sortBy=YYYY

   where XXXX is a CQL style query (see below) of the form <field><relation><term>
   examples: 
            author matches.any "shiva jones"
            title  equals "lord of the rings as metaphor"
            date > "20-jul-2001"
            some contains "three musketeers"
            
         NN is an integer representing the most records you want returned

         JJJ indicates which record in the results to start with (start counting at 1)
   
               example: maximumRecords=20&startRecord=15 would return up to 20 records, 
                           starting from the fifteenth
                                           
         YYYY indicates how the results are to be sorted (defaults to age, newest first)
                  it might have one of the values newer, older, score (relevency), rating, title

The queries

Of the general form <clause> [<combiner> <clause>]*,
              which is interpreted left to right, clause by clause.

            where <combiner> is one of "and", "or", "not";

            each <clause> of the form <field><relation><term>

                    the <field> is one of: some (meaning any of the fields)
                       title, author, date (of posting), format (text, audio, video),
                       subject (categories/keywords), link (URL), summary, 
                       article (body of text + summary)
                      
                       
                    <term> is either <text>, a <date> or an URL
    
                                  <date> is of the form <day>-<month>-<year>,

                                            where <day> is an integer from 1 to 31,

                                             <month> is a three-letter month code 
                                               (one of jan, feb, mar, apr, may, jun, jul, aug
                                                    sep, oct, nov, dec)

                                                <year> is a four digit integer 
                                                      corresponding to the gregorian calendar year 
                                                      (ie 2004)
           
                   <relation> is one of: 
                      matches.any, matches.all, equals, contains (for text fields)
                          >, < = for numeric or date fields
    
               <text> is either a single word, 
                        or a sequence of words separated by spaces, enclosed in
                           quotes (example: "three musketeers" or "united kingdom" 
                                       or "smith jones")

The results

The response to a query is an xml file, similar to the rss headline feed: (Note the addition of fields echoing the query, listing the number of results and the start item of the results, as well as a "score" returned from the search indicating relevency. When numberResults = 0, there may be an tag containing an error message, if an error occured):

<?xml version="1.0" encoding="UTF-8"?>

<rdf:RDF 
  xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
  xmlns:dc="http://purl.org/dc/elements/1.1/"
  xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
  xmlns:search="http://oceania.indymedia.org.au/search/"
  xmlns="http://purl.org/rss/1.0/"
  xmlns:cc="http://web.resource.org/cc/"
> 

<channel rdf:about="http://fred.indymedia.org/search">
   <title>fred IMC</title>
   <link>http://fred.indymedia.org/</link>
   <description>search results from fred imc</description>
   <dc:publisher>Fred IMC Editorial Collective</dc:publisher>
   <dc:date>2004-07-14T04:00:09+0000</dc:date>
   <search:query>items match.any "smith jones" or author equals "John Smyth"</search:query>
   <search:numberResults>4</search:numberResults>
   <search:startRecord>5</search:startRecord>
   <search:sortBy>score</search:sortBy>
   <items>
      <rdf:Seq>
         <rdf:li rdf:resource="http://fred.indymedia.org/feature/display/7256/index.php" />
         <rdf:li rdf:resource="http://fred.indymedia.org/feature/display/6909/index.php" />
         <rdf:li rdf:resource="http://fred.indymedia.org/feature/display/6884/index.php" />
         <rdf:li rdf:resource="http://fred.indymedia.org/feature/display/6815/index.php" />
      </rdf:Seq>
   </items>
</channel>



<item rdf:about="http://fred.indymedia.org/feature/display/7256/index.php">
   <title>Where do Hamilton Centre Candidates Stand on Security Certificates and Secret Trials?</title>
   <link>http://hamilton.indymedia.org/feature/display/7256/index.php</link>
   <description>#media_3245878;left# As of June 2004, five Muslim men have been held a collective total of 144 months in Canadian prisons, much of it in solitary confinement, without charge, without bail, on secret \x93evidence\x94 which neither they nor their lawyers are allowed to see. They have been threatened with deportation to countries where they would be tortured and killed. They are men with wives and children they cannot hug. None has ever been convicted of a criminal offence in Canada. As Federal court justice Andrew MacKay put it \x93\x85in this great city of Toronto, we have our own Guantanamo Bay\x94 
Photo Mahamed Harkat, one of the &quot;Secret 5&quot; detained without charge.</description>
   <dc:creator>Chris Jones</dc:creator>
   <dc:subject>Elections &amp; Legislation</dc:subject>
   <dc:language>en</dc:language>
   <dc:date>2004-06-27T23:02:08+0000</dc:date>
   <cc:license rdf:resource="http://www.creativecommons.org/licenses/publicdomain/" />
   <search:score>100</search:score>
</item>

<item rdf:about="http://fred.indymedia.org/feature/display/6909/index.php">
   <title>Red Hill Tree Sit</title>
   <link>http://fred.indymedia.org/feature/display/6909/index.php</link>
   <description>&lt;img align= &quot;right&quot; hspace = 10 vspace = 10 src=&quot;http://fred.indymedia.org/usermedia/image/2/sit.jpg&quot;&gt; Police, followed by private guards, arrived at the site of the tree sit in the Red Hill Valley yesterday afternoon (June 1, 2004), forcing supporters on the ground to leave rather than be arrested. 

Blasting is scheduled to begin soon to cut an 80 metre wide swath through the face of the escarpment to make way for the controversial Red Hill Creek Expressway.

In the early hours of Sunday, the protesters entered the area and managed to scale the trees and erect sturdy platforms without being detected by private guards posted less than 100 metres away.

Those occupying the trees in a small wooded area just west of Mount Albion Road intend to stay for an extended period of time, up to &quot;several months&quot; they claim, in order to prevent irreparable damage to the escarpment face.</description>
   <dc:creator>John Smyth</dc:creator>
   <dc:subject>Environment : fred politics : Protest Activity</dc:subject>
   <dc:language>en</dc:language>
   <dc:date>2004-06-02T15:14:08+0000</dc:date>
   <cc:license rdf:resource="http://www.creativecommons.org/licenses/nc/1.0/" />
   <search:score>96</search:score>
</item>

<item rdf:about="http://fred.indymedia.org/feature/display/6884/index.php">
   <title>CRITICAL MASS FRED TURNS 7-YEARS</title>
   <link>http://fred.indymedia.org/feature/display/6884/index.php</link>
   <description>&lt;img align= &quot;left&quot; hspace = 10 vspace = 10 src=&quot;http://fred.indymedia.org/usermedia/image/6/critical_mass_feature_pic.jpg&quot;&gt;
&lt;p&gt;fred's CRITICAL MASS bike rides turned the corner on seven-years of cycling culture and action in the city.
&lt;p&gt;CRITICAL MASS in fred started May 1998 with 70 cyclists making their claim to the streets.

&lt;p&gt;Since then, the fred rides have celebrated the world's best-loved invention - the bicycle -with monthly rides on the last Friday of each month departing from Hess and George Streets in Hess Village (5:30 p.m.).
&lt;p&gt;&lt;a href=&quot;http://fred.indymedia.org/en_US/resize.css/newswire/display/6880/index.php&quot;&gt;[Account of May 2004 Mass ride (incl. photos and audio)]&lt;/a&gt;</description>
   <dc:creator>Randy Smith</dc:creator>
   <dc:subject>Protest Activity : Transportation</dc:subject>
   <dc:language>en</dc:language>
   <dc:date>2004-05-30T14:36:13+0000</dc:date>
   <cc:license rdf:resource="http://www.creativecommons.org/licenses/by-nc/1.0/" />
   <search:score>86</search:score>
</item>

<item rdf:about="http://fred.indymedia.org/feature/display/6815/index.php">
   <title>Waterwatch Pays City for freedom of Information</title>
   <link>http://fred.indymedia.org/feature/display/6815/index.php</link>
   <description>#media_3245745;right#fred \x96 At a City Hall media conference Friday May 21, fred Water Watch, a pro-public water coalition announced a substantial contribution of $1072 to Councillor Sam Merulla\x92s Freedom Of Information (FOI) request seeking further financial disclosure on the City\x92s current private water contract.</description>
   <dc:creator>waterwatch jones</dc:creator>
   <dc:subject>fred politics</dc:subject>
   <dc:language>en</dc:language>
   <dc:date>2004-05-23T00:59:12+0000</dc:date>
   <cc:license rdf:resource="http://www.creativecommons.org/licenses/publicdomain/" />
   <search:score>60</search:score>
</item>


<cc:License rdf:about="http://creativecommons.org/licenses/publicdomain">
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction" />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution" />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/by/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/Attribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction" />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution" />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/by-nd/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/Attribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction" />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution" />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/by-nd-nc/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/Attribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:prohibits rdf:resource="http://web.resource.org/cc/CommercialUse " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/by-nc/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/Attribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
  <cc:prohibits rdf:resource="http://web.resource.org/cc/CommercialUse " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/by-nc-sa/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/Attribution " />
  <cc:requires rdf:resource="http://web.resource.org/cc/ShareAlike " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
  <cc:prohibits rdf:resource="http://web.resource.org/cc/CommercialUse " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/by-sa/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/Attribution " />
  <cc:requires rdf:resource="http://web.resource.org/cc/ShareAlike " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/nd/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/nd-nc/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:prohibits rdf:resource="http://web.resource.org/cc/CommercialUse " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/nc/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
  <cc:prohibits rdf:resource="http://web.resource.org/cc/CommercialUse " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/nc-sa/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/ShareAlike " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
  <cc:prohibits rdf:resource="http://web.resource.org/cc/CommercialUse " />
</cc:License>

<cc:License rdf:about="http://creativecommons.org/licenses/sa/1.0/ ">
  <cc:requires rdf:resource="http://web.resource.org/cc/Notice " />
  <cc:requires rdf:resource="http://web.resource.org/cc/ShareAlike " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Reproduction " />
  <cc:permits rdf:resource="http://web.resource.org/cc/Distribution " />
  <cc:permits rdf:resource="http://web.resource.org/cc/DerivativeWorks" />
</cc:License>

</rdf:RDF>

-- RomanO - 14 Jul 2004

Topic revision: r2 - 30 Sep 2005, AlsteR

Local

Home

All sections
Global network
Local IMCs
Tech
Sandbox

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback