doug: Worked for me - hi!
doug: I wonder why irc.indymedia.org is down
doug: The wiki isn't working for me either.
vic_keith: kropotkin is down... don't know...
doug: It's a pisser because we were working on a big document there that we wanted to share for the meeting.
doug: Maybe I should just post it to the list, but it would be so much easier to read in the wiki.
vic_keith: yup... unless you know of another wiki..
doug: I don't...
vic_keith: maybe just post it to the list then
doug: Okay, done.
\x95scott-van has joined #active2
\x95mike has joined #active2
mike: hi guys.
mike: irc.indymedia.org works for me...
doug: hey mike, scott
mike: so, is there an agenda?
mike: (this server seems slow :()
vic_keith: not afaik. kropotkin is down
vic_keith: mike: best i could find short notice.. know of a better one?
mike: did anyone check out http://meejah.homeip.net/~mike/active2-june-4-2002.tar.gz ?
mike: vic_keith no, just whining
vic_keith: mike: not yet.
mike: ...has anyone else tried irc.indy just now, though? it's up for me...
doug: Mike: i did
mike: and no dice?
doug: Do you have it up and running anywhere?
mike: doug no; it's more for showing how cheetah stuff might work, and a sort of concept of how the code might be layed out...
doug: Not knowing python, I couldn't really follow
doug: The templates were easy enough to look through.
vic_keith: i'll take a look.
mike: doug: yes, the cheetah stuff looks like it might be very nice
mike: ...uhm, i was just checking out irc.indy, and I'm not convinced it's really the indy IRC server; there's a billion channels and no #tech or #indymedia
doug: Not good!
vic_keith: mike: r u using mirc?
mike: no, bitchx
mike: should the IP be 18.104.22.168?
mike: ...anyway, that doesn't matter. what's on the agenda for today?
mike: should we discuss doug and anton's gui work?
vic_keith: i don't really have anything to add to an agenda.
vic_keith: i want to look at your python sutff furst.
mike: yeah, okay
doug: there's also the documentation I just posted to the list
mike: i'll put it in CVS on ender soon
mike: it's in CVS on my machine; see the CVS-INSTRUCTIONS file in my home directory on ender (/home/mike)
mike: doug: haven't seen that; you mean the architecture stuff?
doug: Yeah, it's on the wiki which is down so I just posted it to the list.
mike: i haven't taken a close look at it yet.
doug: It's loooong
mike: oh, okay; i've got it now
vic_keith: doug: nice work..
doug: Helps explain the gui, though
mike: we should discuss possibility of ``internal'' Text representation...
mike: so, agenda: #1 GUI, #2: internal Text representation
mike: ...is someone logging this?
vic_keith: i am
mike: okay, cool
mike: does anyone have to leave at a specific time? (server is down at work, so i'm at home just now; no contraints on me until server back up, likely in a couple hours)
vic_keith: not really...
doug: I'm free
mike: should we go over #2 first? probably it's the shortest...
mike: basically the idea is this:
mike: if we have an ``internal'' format/method to represent Text, then we can accept Text content in any format so long as somebody has written a parser which converts it into the internal format
mike: then, we use output filters to conver the internal format/method into whatever is needed (i.e. HTML mostly, but also RSS, plain text, PDF, etc.)
doug: what other input formats are you talking about?
vic_keith: that sounds good
mike: upshot: we don't have to worry about thigns like ``bad'' html tags or the like
mike: doug: at least HTML, text-only and probably PDF
mike: upshot: many output formats are possible
mike: upshot: easy to add more input/output formats
doug: How are we representing this internally? What are the 'fields' of the date?
vic_keith: what do you think this format should look like? newsml?
mike: vic_keith no, newsml is basically meta-data
mike: i would advocate a python object similar to what's in the code i mentioned earlier
mike: basically, it's a stream of python objects representing the various things in the text...
mike: ...basically, i would say we just need to define a data structure for whatever language the implementation will be in and use that
mike: making an XML DTD or similar will be more pain than it's worth, IMO
doug: you mean like ...text...link...text...email...?
vic_keith: sounds good. are you looking at persistance, or will this be an "object stream"?
mike: doug: sort of, yes: there's a Text base class for things which are text (text, link, email, etc.) and other things, too (like Paragraph, Document)
mike: vic_keith no, no persistence.
vic_keith: i like it. python has some great OO features
mike: vic_keith basically, an input-filter will produce the object stream from it's data (text, pdf, html, whatever) and then it will get passed around inside. possibliity to cache it, of course...
mike: vic_keith agreed on OO/Python
mike: vic_keith take a look at the code-example i made; it will probably be more clear what i mean
mike: ...although it's pretty straightforward just now
vic_keith: cool.. will do
mike: ...look in python/active2/document/*.py
mike: ...look in python/indymedia/document/*.py
doug: We might support that format from the newsml folks... itf something or other...
mike: the second one
mike: doug: which do you mean?
mike: the newsml example i looked at all had their own simple-DTDs for the actual content; newsml was for the meta-data it looked like to me...
mike: or sometimes they just linked to the ``actual'' content (PDFs or whatever0
doug: Not newsml - the same people had also put out a format for the text.
mike: doug: oh, okay. certainly worth taking a look at, I suppose
mike: will anyone actually want to submit things in this format, though?
doug: Found it - it's the NITF format.
mike: what i was thinking is that authors can use their favourite editor to produce HTML or LaTeX or whatever and then send that in
doug: News Industry Text Format
vic_keith: mike: doubt it
mike: doug: do you have a URL?
doug: I've posted it to the list before, I think it might be in the documentation too... I'll look
mike: for the first version, let's just support simple things (text-only, HTML would be my votes)
mike: ...we should make it easy to write new parsers, of course, after which we can write parsers for whatever seems to be a popular but unsupported format for submissions
mike: I have made just a text-parser which is pretty simple and looks only for titles in the text (title vs. paragraph)...
mike: python has built in HTML parsers, too
doug: duh - http://www.nitf.org
vic_keith: mike: yup
vic_keith: SAX, expat
mike: vic_keith yes, it can use those, too
vic_keith: mike which do you prefer?
mike: ...but there's a pure-python HTML parsing module as well
mike: expat is pretty fast and good, IME
mike: haven't done too much with XML, though
\x95wilmer has joined #active2
mike: (mostly with the XERCES stuff from Apache...)
vic_keith: sorry i was thinking xml, not html
mike: vic_keith well, html is XML
wilmer: sorry guys....i don't know how to work IRC well
mike: anyway, the pyxml stuff can use expat if it's installed
doug: hi wilmer
mike: the pyxml module is really neat that way: it tries to use the ``best'' xml parser you have installed, and falls back to a pure python one
vic_keith: are we done on this? wanna talk about the GUI?
mike: yes, okay, so we've decided we need some sort of internal representation?
wilmer: hi doug
vic_keith: yeah. i like the py objects
mike: ...most likely a python object-stream if we use python?
mike: okay, sounds good. yes, let's move on to #1
mike: doug: quick peek at NITF DTD looks like it's sort of HTML with a whole bunch of other stuff for copyrights, authorship, revisions, and other meta-data-ish things...
doug: In the docs where it says Abstraction of Storage/Retrieval - this is touching on internal representation.
doug: Mike: yes, it has stuff that's really part of our article model.
doug: There's overlap between newsml and nitf
mike: does anyone actually use NITF? like reuters or something?
doug: yeah, it was their first stab at xml and is the first revs of a lot of systems.
vic_keith: yah... newsml and nitf are competing standards
vic_keith: i think
doug: they are put out by the same org
mike: uhm, GUI: should we go page by page?
doug: We used nyc.indy content for some realism
mike: what is the articlelist on the right?
vic_keith: heh... nice
mike: is that just raw chronological list of all articles?
doug: yep the 'raw newswire'
doug: ignore 'latest by keyword'
mike: should there be a way to switch which newswire is shown there?
doug: The home page newswire shows all the articles
mike: maybe a little choice box in the title-bar?
mike: what if a user doesn't want to see all the garbage, but wants just ``local'' content or ... whatever
doug: newswire list on left gets you to other newswires
mike: oh, i see it.
doug: mike: they could have this on their 'personal newswire'
mike: i'm thinking along the lines of the slashboxes on slashdot; on may change which ones show up if they like. maybe the ``default'' right-bar newswire should be similarly changable
mike: that is, the default would be ``raw'' but one could go and change their preferences to make it show ``my newswire'' or ``my friend's newswire'' if they like...
doug: I think we decided to push this sort of customization onto the personal newswire
vic_keith: yeah... instead of "article list" it would say the name of the current newswire
mike: doug: where is the personal newswire screen? does it have the features column?
mike: what i'm thinking here is that for repeat users, they might not want to see the raw article-list ever again...
mike: oh, okay. so is the center column still ``features'' or whatever? or is that stuff the user has manually put into ``my newswire''?
doug: mike: could be either
doug: The idea of a newswire has changed a lot.
doug: It now covers the idea of groups/newsdesks/editorial filters, etc...
mike: hmm. okay, i guess it doesn't really matter, then; they could go to the ``my newswire'' page all the time (instead of the front page) and still see things that the editorial collective has put into the ``features'' newswire, say?
doug: The personal newswire is like any other newswire except we have 'automatically' put particular items on it.
doug: mike: yes, we could set it up like that.
mike: okay, sounds good
doug: The home page of the site is also implemented as a newswire.
vic_keith: what do you mean by home page
mike: doug: yes, that's a good idea
mike: ..saves code, too
doug: home page == utopia.indymedia.org
mike: vic_keith utopia.indymedia.org or whatever
\x95BlackDeathX has joined #active2
doug: mike : yes saving code is big priority
\x95BlackDeathX has left active2
doug: whao, someone walked over our graves...
vic_keith: hah. i like the idea of "everyhting is a newswire"
vic_keith: code reuse is cool
doug: There are just article pages and newswire pages and admin pages
mike: ...and user pages and the publishing stuff
mike: but yeah, i get the idea
mike: sounds good
doug: user page is a newswire and publishing is what I'm calling admin
mike: oh, okay
mike: what about user-preferences?
doug: was lumping that under admin. or do you mean visitor view preference (session data)
doug: Things like language options?
mike: yes, that sort of thing; options which don't require a login
mike: (i.e. look of the page, maybe, or language or high vs. low bandwidth or whatever)
vic_keith: visitors would have session data, users would have permanent prefs (i think)
doug: Was hoping to make most of the implicit in the navigation
doug: ...most of that implicit...
doug: It's not there yet.
mike: okay, no problem
mike: any other questions/comments about front page?
mike: ...i really like the ``who's responsible'' stuff at the bottom
doug: Oh, buried in the docs is a site map we worked on for a bit:
doug: We haven't given much thought to themes - in fact I forgot to put it in the docs!
mike: themes are kind of back-end issue, no?
vic_keith: not a big deal. aesthetics.
mike: i mean, basically it means producing a different set of Cheetah (say) templates
mike: ...and then using a cookie-based option to pick which directory to grab templates from
vic_keith: ummm... in its simplest sense it means diffrent style sheets
doug: yeah, but we should be building the interface to choose which template.
mike: (we should consider this when architecting the web stuff...)
vic_keith: yup worth thinking about
doug: We're trying to get everything in the GUI if we can.
mike: vic_keith well, that's just a subclassed template with a different line...
mike: with the cheetah stuff, we can have the pure-python base class define a stylesheet(self): method, which imc's could override (or a sub-template could override)
doug: The themes wouldn't be changing a whole lot because the coloring many of the bars is up to the newswire editors.
mike: they're really quite flexible, and the templates themselves are object-oriented
mike: doug: well, perhaps. i can imagine that different IMCs are going to perhaps want quite different-looking sites...
mike: anyway, it's a sort of ``middle-end'' issue (i.e. template configuration, basically...)
doug: Yeah, at the IMC level they get to write the templates.
mike: the cool thing about cheetah is that they can just subclass the templates they don't quite like...
doug: At the newswire editor level they get to set the colors of some of the bars.
mike: ...and then just re-write the particular things they don't like...
mike: (so long as we design the base templates nicely)
vic_keith: should we be looking at outputting strict HTML 4 or XHTML?
mike: vic_keith no opinion, really; that is a template issue, too (sort of).
mike: ...we could make a different set of templates for each...
vic_keith: tru dat.
mike: (that is one set which does strict HTML 4, one for HTML 3 or ...)
vic_keith: heh.. how standards conformant do you want your independent media ?
vic_keith: that would be a hot user-pref
doug: can we look at the newswire admin page? http://www.syntropo.com/Utopia/admin_newswire.html
mike: neat. i see.
mike: looks good
doug: This one interface would be used for editing the home page, user page, and each newswire
mike: yes, that sounds fantastic
vic_keith: be nice if we could add an RSS news feed from anywhere
mike: i like that you can add an article or an article-list
mike: vic_keith: could that just be an article-list?
doug: Each newswire is an RSS feed!
mike: doug: yes, that's good
vic_keith: very cool... i like
mike: so like http://imc/rss/newswirename.xml or whatever?
doug: Click on one of the article list links
mike: (to get the rss feed itself, i mean)
doug: mikeL yes
mike: doug: the newswire-list editing looks great, too
vic_keith: very nice
mike: (i can think of lots more ``automated'' options; we should write those as modules...)
vic_keith: this is the friendliest UI i have ever seen on an indymedia site
mike: e.g.: regular expressions for various fields, media-types (limiting, searching),
vic_keith: mike: yes... i think that we should set it up so that lots of different compnents can be created as modules (say, the local weather)
doug: mike: yeah lots of automation possible, but could be a drain?
mike: doug: well, like you said, the administrators should be able to add/remove modules
doug: keith: exactly! We can make new modules later that can do lots of things.
mike: if there's no ``newswirelist.regular-expression'', then they don't get that option...
doug: Notice that you can also drop in a feedback form (look at the add... form on the newswire edit page)
mike: (that's just an example, of course)
mike: oh, i see. yes.
mike: these could also be ``modules'', so keith's example works (i.e. add....weatherbox)
doug: yep, add weatherbox! add realtime chat box!
mike: yeah, this sounds great
doug: wow - I'm so glad you guys 'get' this!!!
mike: now we just need the backend flexible enough
mike: shouldn't be a problem though
doug: yikes, I know. I think it's doable.
mike: ...random implementation thought: for example, the interfaces can be cheetah-templates which are included in the final template as needed
mike: (that is, if the module is enabled)
mike: (bear in mind: templates can be included/subclassed all over the place, so the ``about this newswire'' area can be a separate template, for example)
doug: yeah, I was thinking of it in terms of having a 'template module' for the 'newswire item' module that gets loaded when an item of that type is edited.
vic_keith: sounds good
doug: I think we still need to work through just what a 'module' is - see the docs I posted...
vic_keith: we'll have to define a module interface in the backend
mike: doug: yes, true
doug: Did you do that at all in your python stuff, mike?
doug: Modularize things?
mike: no, not yet
mike: there's not that much stuff yet
mike: ...but the stuff that is there is in ``modules'' i suppose
mike: (for example you can take out a parser without mayhem happening)
doug: I did it with the simple little demo I put up months ago.
mike: i guess what we need to really decide upon is what the modules' role will be
mike: ...will it just do one thing (i.e. be an interface for administering the newswiree-list)
mike: ...or will it instead be everything to do with newswire-lists?
mike: that is, provide the interface and the required backend functionality.
mike: i would say the latter...
mike: ...although there's a decent case for the former, in some instances
doug: i think it's a mix
mike: (i.e. maybe you want the functionality but don't want to provide the interface; the interface could be turned off as a configurable option even in the second case)
vic_keith: i think that we need to separate backend and presentation...
mike: vic_keith well, true. i'm thinking more along the lines of how is a ``module'' defined
doug: we have to define what it is a module does (and I think there should be several kinds of modules) and what it is the 'core system' does.
mike: doug: well, won't modules need to plug into the ``core'' system? and the ``interface'' system?
mike: so, yes, i guess we need to define exactly what the limits of the ``core'' are...
mike: ...also, we would need to decide where to put hooks for modules
mike: does anyone know emacs lisp (to illustrate examples)?
doug: I outlined the types of modules I thought we might need in this post:http://lists.cat.org.au/pipermail/webcoders/2002-February/000233.html
mike: okay, i'll re-read that
mike: okay, i have output and input modules in my example code
vic_keith: mike: that's mostly how i see it: input and output modules.
mike: ...and abstraction modules
doug: briefly it's input, output, responders, abstractions, templates, newswireitems
doug: six in all
vic_keith: nice.. will take a look.
vic_keith: i gotta go. low serotonin...
mike: i'm not clear on what the responder modules are for
mike: (i'm not clear why they should be separate)
doug: They respond to url essentially
mike: if some module needs user-input, shouldn't it handle parsing the request?
vic_keith: i'll leave my irc client on and post the logs later. ciao.
doug: very high level - for inventing new functionality later
mike: doug: oh, i see: so a particular responder-module gets to ``claim'' a particular url-type?
mike: okya, bye keith
doug: newswire and article would be responder modules
doug: bye keith
mike: (that is, the ``content server'' module would grab ``/filekey/ljasjdlljkafsdljkasdf'' urls?
mike: while the article-out module might graph /article/ urls?
doug: yeah, article responder module would return the article
mike: okay, that makes sense
doug: each page in the demo would be a responder module!
mike: so someone could make their indy node into a content-server by enabling the content-server module which would grok /filekey/jkldfasljk urls...
doug: Yeah, I guess you could turn off the article module.
doug: A lot of other things might rely on it. I guess I was thinking there would be a set of modules that could be overridden, but not turned off.
mike: so the ``newswire'' module would provide the newswire-admin pages?
doug: there would be a separate module for newswire and 'admin newswire'
mike: doug: i see what you're thinking
doug: they might be organized into different 'module packages'
mike: i think i was envisioning something similar
doug: for instance the newswire module package would have the responder module and the template module.
mike: modules would correspond to python packages (for me), and imc's could subclass them
mike: doug: yes, that sounds about what i was thinking
doug: Yes, I was thinking of everthing being subclassable like this
mike: so the config would boil down to specifying which python-class is the ``admin module'', for example
mike: (it would need to be subclassed from the ``base'' admin-module, but...)
mike: one could just specify ``None'' to turn something off...
doug: Yep. I was hoping the config would all boil down to turning module and module packages on and off.
mike: doug: yes, the interface could easily look like that; i was just thinking of the actual config-files
doug: I had been thinking (in PHP) of the config file being written by this sort of interface. It would contain complex array structures indicating what was on/off.
mike: ...that is, when an IMC decides they're going to subclass the admin module becuase they dont' quite like the interface, then they just change the name of the ``admin interface'' class in a config somewhere
doug: I was hoping to do away with hand edited config files
mike: okay, similar minus the complex structure...
mike: doug: well, someone has to write some code at some point
doug: That's us
mike: i'm thinking about when some tech guys at a different IMC want slightly different code...
mike: yes, if it's all stuff we've provided there shouldn't be any hand-editing of anything
mike: (once apache is up and going with active2, that is...)
doug: So am I. I was hoping to force them to write stuff to our modular system so that it could be easily shared.
mike: doug: but how would the config system find out about their new code, unless they tell it where it is?
doug: When you visit the 'module admin' page it would look in your modules directory.
mike: yes, it could auto-import everything there and tell you which things have the right inheritence
doug: Yep, that's the idea.
mike: (that is, if your class FooBar inhereits out of the newswire-admin stuff, then it can be specified as the newswire-admin module to use...)
doug: I had been thinking that modules were objects
mike: yes, such a GUi would be pretty easy to do
mike: yes, modules are objects (in python). uhm, this might get complicated if you don't know OO
mike: do you?
mike: yes, modules are objects. they have methods which return instances of classes
doug: I had all my modules inheriting from a base module class
mike: okay, that works
doug: and also from a class that defined the module type (and API) for that module.
mike: so if we have a indymedia.active2.modules.newswire.admin or whatever, then people can subclass the classes in there if they want
doug: There was a class for each module type.
mike: oh, okay, i think i'm misleading you
mike: i was thinking in terms of python-modules for a second there
mike: yes, we can have a ``module'' class which is subclassed for the various module-types
mike: ...and it's very easy to check inheritence, etc., of things in the IMC's ``extra modules'' python-area
doug: Once this is done then all we have to do is write modules!
doug: you mean in your python code?
mike: no, it's not done now
doug: Still lots to do.
mike: yes. we'll have to think carefully about the exact module interfaces
mike: (that is, the public methods available)
mike: i'll sketch out more of this before next week, so we have something concrete to discuss
doug: Yep, I was hoping to have it so standardized that modules could rely heavily on each other.
mike: (i.e. a class tree)
doug: That would be great!
\x95wilmer has left active2
mike: probably it will change drastically as implementation happens and limitations are found, but it's good to have a start
doug: I totally agree, it'll change and we should start
mike: okay, so that's my big TODO for this week.
mike: have you looked any more at templates? found different systems, maybe?
mike: i really like the cheetah stuff from what i've seen so far, but maybe there's better things out there...
doug: I was thinking we'd just roll our own.
doug: But if we can use something out there, great.
mike: cheetah is a templating infrastructure; it doesn't presume anything about what you're templating
mike: for example, it can be used to make templates which produce LaTeX or HTML or XML or ...
doug: In general I like the idea of abstracting any 'outside' technology so we can override it easily.
doug: If cheetah does it the way we like, then let's make that the API for templating.
mike: the other nice thing about it is that you can have pure-python base classes
mike: so most of the work can be done outside of the actual template, hiding all the nastiness from Web designers...
doug: I was thinking that html/rss/xml options would be handled by the output modules.
doug: Output modules would use template modules.
mike: the idea is that it should be easy for designers to change the templates, and easy for programmers to change the other stuff...
doug: Yes, totally agreeing with the idea
doug: There should be no html code in modules except from template modules.
doug: except for...
mike: yes, probably
mike: there might be utility classes the templates would like to call to get their output
mike: ...but yes, all the HTML stuff should be easily accessible to designers
doug: And if the HTML is all contained like this then we could also output it all in other formats. I was thinking of something like newswire admin responder module outputing in XML-RPC to a flash client....
mike: yes, other formats should be easy to do, though; we shouldn't presume HTML
mike: ...work servers are back up now, so i should go soon
doug: My puppy needs to run too, good timing.
mike: okay, thanks for meeting
doug: was it you who was archiving?
mike: we'll hopefully be closer to starting ``real'' first-try at implementing stuff next week
mike: ...no keith was
mike: he said he left himself logging, though...
doug: oh yeah, there he is.
doug: Yeah, next week. It would be nice to get your stuff looking like our stuff.
mike: well, hopefullt the logs will be up after kropotkin is happy
mike: doug: yes, i was converting the index.html to an index.tmpl
doug: Keep in mind we still need to work on these a lot.
mike: i'll sketch out more robust example code and maybe make a news.tmpl for next week, with some ``real'' content from the backend (which sort of works, too)
mike: yes, i realize they're not final. i won't spend a whole bunch of time getting the html right
doug: Is it hard to update your templates with new designs from us?
mike: there's an example template in the code i posted the link to; check it out
mike: for example, instead of ``Utopia'' you just need $imcName
doug: Yeah, I was looking through it.
mike: anyway, it was just a quicky
mike: i'll bear in mind ease of updating...
mike: ...that will be important for a production system, too
doug: Maybe you could send back the template you make out of our stuff for us to see.
doug: We could try to edit that?
mike: okay. hopefully it renders decently in a web browser without the backend stuff doing it's substitution...
mike: yes, ideally.
doug: Okay, great chatting with you.
mike: ...but go ahead and make any changes you need to for now...
mike: yeah, you to.
mike: see you next week...
doug: okay, bye