Plone notes from Raik

Some months ago, I set up a plone site for our group -- http://sfiles.embl.de .
Mac asked me to blog my installation notes, so here they come:

The basic steps are:
- create a zopeuser on your server
- install python and zope
- install plone as a product into your new zope server
- create a plone instance
- optional: put it behind apache

Note: Some things may have changed with the more recent versions of Plone / Zope. These are just my onw quick-and-dirty notes. I hope they are still helpful.

Initial setup
=============

- Create zopeuser

- add zopeuser to sudo list without password
in /etc/sudoers add::

zopeuser ALL=(ALL) NOPASSWD: ALL

Python
======

- download 2.4.3 from python.org

tar xvf Py*
sudo mv Python-2.4.3 /usr/local/src
cd /usr/local/src/Python-2.4.3
./configure
make
make altinstall ## don't override default python

-> creates /usr/local/bin/python2.4

## local installation: ./configure --prefix ~/data/local

Bibutils
========

optional, only needed for the literature management

- download bibutils_3.27_i386.tgz from
http://www.scripps.edu/~cdputnam/software/bibutils/

sudo mv bibutils_3.27/* /usr/local/bin/

Subversion
==========

sudo yum install subversion

Zope installation
==================

- install Zope from source, download untar, then...::

mv Zope-2.9.3 /usr/local/src
cd /usr/local/src/Zope-2.9.3
./configure
make build
sudo make install
cd /usr/local/Zope-2.9.3/bin

mkdir /usr/local/Zope-2.9.3/instance
./mkzopeinstance.py
folder: /usr/local/Zope-2.9.3/instance \
user: admin

- make sure Zope folder belongs to zopeuser

- check $PYTHONPATH
- start zope server::

cd /usr/local/Zope-2.9.3/instance
./bin/zopectl start
>>> . daemon process started, pid=26472

- log into ZMI::

http://sfiles.embl.de:8080/manage


##local installation::

./configure --prefix ~/data/local/Zope --with-python ~/data/local/bin/python2.4

Install Plone
=============

- download Imaging-1.1.6b1.tar.gz from http://effbot.org/downloads/#Imaging::

scp 'grunberg@kaa.embl.de:data/input/Imagin*' ~/input
untar...
sudo mv Imaging-1.1.6b1 /usr/local/src/
cd /usr/local/src/Imaging*
sudo python2.4 setup.py install

- download Plone-2.5.tgz and install all needed Zope products::

gzip -d Plone*
tar -xvf Plone*
sudo mv Plone-2.5 /usr/local/src/
cd /usr/local/src/Plone-2.5
emacs CMFPlone/INSTALL.txt &

rm -r Five ## for Zope 2.9.x
rm *.txt

mv * /usr/local/Zope-2.9.3/instance/Products/

- restart Zope
in ZMI/Control Panel: Shut down Zope
in /usr/local/Zope-2.9.3/instance: ./bin/zopectl start

- install Plone site
in ZMI/ (root folder): Add/Plone site
ID: project

- check: Plone site should be accessible under
http://sfiles.embl.de:8080/project

put Zope behind apache
======================

- check that "Virtual hosting" exists in ZMI root
- work through
http://plone.org/documentation/how-to/plone-with-apache-1.3

in short:

- add to /etc/httpd/conf/httpd.conf::


ServerName synplexity.embl.de
ServerAlias www.synplexity.embl.de
ServerAdmin webmaster@synplexity.embl.de
ProxyPass / http://localhost:8080/VirtualHostBase/http/sfiles.embl.de:80/projects/VirtualHostRoot/
ProxyPassReverse / http://localhost:8080/VirtualHostBase/http/sfiles.embl.de:80/projects/VirtualHostRoot/



Deny from all


- (re)start apache::
sudo /usr/sbin/apachectl graceful

- check: Plone site now accessible under
http://sfiles.embl.de/

- direct access to Zope w/o apache is still possible with
http://sfiles.embl.de:8080/sfiles

Now, the plone site should be up and running behind your apache server. The remaining installation log describes the installation of some additional plone plugins. I'll just give the list of plugins I found useful:

- ATExtensions -- needed by most plugins (perhaps included in recent plone versions?)
- CMFBibliography -- very neat literature management
- AmazonTool -- to import book citations from Amazon
- ATBiblioList -- create virtual literature lists

I put up a little first-steps page on sfiles to get users started:
http://sfiles.embl.de/help/first_steps

There are some more notes on customizing sfiles and create custom workflows-- I can post them later if you are interested.

That's it... questions / comments are welcome.
Good luck!
Raik

reinstalling drupal, preparing for live & dev sites

Ok, I'm archiving my current dev install of drupal (which is bloated with modules, some of which have altered the mysql database) and starting with new install of 5.1. Once I have just the right constellation of modules locally, I'll copy them over to our live site (parts.mit.edu/igem07). Here are the mysql commands I used to reset the database for my new drupal install:

• mysqldump -u username -p --databases drupal >/tmp/drupal.sql
• drop database drupal;
• create database igem2007;
• GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES
  ON igem2007.*
  TO 'drupal'@'localhost' IDENTIFIED BY 'foopass';

Then I grabbed drupal 5.1 and followed the install instructions to unpack the files and move them into my webserver's base directory. So, from the terminal:

• cd /Library/WebServer/Documents
  
• rm -rv *
• curl -LO http://drupal.org/files/projects/drupal-5.1.tar.gz
• tar -xzvf drupal-5.1.tar.gz
• mv drupal-5.1/* drupal-5.1/.htaccess ./
• rmdir drupal-5.1
• chmod 777 ./sites/default/settings.php
  - this part isn't actually mentioned in the install.txt; see my earlier post
• visit the drupal site root via web browser and supply database info
• chmod 644 ./sites/default/settings.php
  
• mkdir files
• sudo chown www:admin files
• sudo chmod 755 files
  

That's it! Drupal 5.1 has been restored to virginal status. I'll post the modules I decided on and the configuration details next.

References:

• MySQL:
• a handy list of MySQL commands (pantz.org)
• MySQL Usernames and Passwords
• MySQL config instructions for drupal
• Downloading files with curl (ONlamp.com)
• Using tar to archive (cs.duke.edu)
  

An Open Scientific Future (or, an email to OWW)

"If you had five minutes on stage what would you say? What if you only got 20 slides and they rotated automatically after 15 seconds? Would you pitch a project? Launch a web site? Teach a hack?"

Check out Deepak Singh's presentation at Ignite Seattle, An Open Scientific Future.

I liked his observation that scientists already have pretty good access to information via tools like ncbi, but to use sites like that you really have to know what you're looking for quoting (he showed a quote by Jon Udell: "Effective search depends on reservoirs of tacit knowledge and conscious skill. Some people possess much deeper resevoirs, and/or can tap into them more effectively, than others. That makes them valuable.")

He then said "I do know what I'm looking for but I can't share that information with the rest of the world, it's limited to me. And that's the challenge, and that's why science needs to get open. Because historically it resided inside us completely." He then talks about some current examples of systems that allow the community to share that expertise in finding data.

Also, Austin and I just checked out his blog and noticed his post about OWW and that OWW has now got a technorati tag. Also also, I just noticed that someone named pedro is mentioned on deepaks blog as interviewing Jason, so Jason, is this all old hat to you?

second to the right, and straight on 'till morning

Seeqpod is cool. I would love to find (or build, if I could) a web app that let me and a group of friends chat while listening to and collaborating on a collective music playlist. (Could ajax be used to stream the current song from the person who had added it to everyone else's browser?) I constantly want to share what I'm listening to with my friends and get their reactions, and listen to the music they think I would really dig. I can imagine keeping one browser window open on the collaborative list and just listening all day (gosh, maybe the site could watch a user's last.fm page to see what they've scrobbled recently and suggest that as the next song in the list.)

I don't know if I'm just paying more attention to what's happening online these days, or if it's actually the case that the rate of the creation of interesting and sophisticated web services & applications is accelerating. It seems like every day a new one is on someone's radar, remixing and building from what was on the radar yesterday. It's bewildering, but it's wonderful too, and it's the sort of thing presaged by ray kruzweil and everyone in the "Singularity" camp.

Or is this just the inevitable feeling of getting old? Take solace, for the internet is humanity's Neverland.

setting up parts.mit.edu/igem07

Also see the earlier post on setting up drupal 5.0 in osx 10.4.

• Fixed permissions on settings.php:
  chmod 755 sites/default/settings.php

• setup crontab (also see here):
  crontab -e

  # m h  dom mon dow   command
  55 23,5,11,17 * * * /usr/bin/wget -O - -q http://parts.mit.edu/igem07/cron.php

• Setup files directory (Note: on osx apache runs as user 'www'. It's different on our server, which is running some flavor of ubuntu/debian: the user is 'www-data'. Find apache's username on your system with 'ps aux | grep apache'.)
  sudo mkdir files
  sudo chown www-data:admin files
  sudo chmod 755 files
  

Building Blocks

To summarize the Features of iGEM2007.com:
Rich user & team profiles; Team blogs, Forums, Wiki; Team-centric user access control; Google maps + teams mashup; RSS for everything; Limited, distributed (amongst team leaders) user registration; Feedback & submission of team application; Help docs based on Books, oh yeah, and tagging and digging, of course.

How do we build it? Hopefully a functional skeleton will coalesce out of the following list of drupal resources:

practical demonstrations & explanations

• Great list of modules useful for building "YahooGroups-like mailinglists and wiki support," (includes 4 og related modules) AND a 10 minute screencast of how to set it all up. Perfect? Maybe - thanks Zack Rosen!
• Another great screencast from Rosen demonstrating "how to use Drupal to create Google Maps mashups of virtually any arbitrary data or content with no coding in minutes." Uses Views, CCK, Node Import, GMaps, & Location modules.
  
• Dojo 1 - Drupal basics, FormAPI, & CCK: writeup & screencast
  
• Dojo 3 - Concluding Dojo 1, NodeAPI: writeup & screencast
• Dojo 4 - Views, Panels, & CCK: writeup & screencast
• Dojo 6 - Drupal 5 Themer Pack: screencast (no writeup yet)
• 20+ screencasts listed on drupal.org
  

modules

• Overview of Drupal 5 components & example modules - from api.drupal.org doc
• Creating Modules (a tutorial): Drupal 5.x - "The full tutorial will teach us how to create block content, write links, and retrieve information from Drupal nodes."
• php demonstrating how to view a list of nodes on a google map using the views, location, and gmap modules
• Gmap - API lets other modules use google maps, can generate basic maps in any node from gmap macro code. Integrates with the location.module. Port to 5.1 still unstable.
  
• Location - API for geocoding (only u.s. and some of ca & de?), can add location fields to nodes, can do proximity searches on nodes. Port to 5.1 still unstable.
• Views - "provides a flexible method for Drupal site designers to control how lists of content (nodes) are presented... This tool is essentially a smart query builder that, given enough information, can build the proper query, execute it, and display the results."
• CCK - "allows you add custom fields to custom content types using a web interface. In Drupal 5.x, custom content types can be created in Drupal core, and the Content Construction Kit allows you to add custom fields to any content type." Here's the list of all Dupal 5.x CCK-related modules
• Node Profile Module - "This module builds user profile's as nodes, which ... allows using CCK and its field types as well as the CCK form builder. For a maximum of flexibility it'll be also possible to use custom node-types and modules instead of the CCK." Also, it's integrated with views.module.
• Organic Groups (og) - "An organic group is created by a single group owner, who has special permissions including the ability to delete the group the owner created. Group subscribers communicate amongst themselves using the group home page as a focal point." - basis of "team portal" pages? Or could it just be done with a taxonomy? Also see og list manager and og views. Alternatively, see mailhandler for a non-og way to link forums w/ lists.
  
• Converting 4.7.x modules to 5.x

themes

• How to theme CCK input forms - "So, as a non-developer, semi-technical, marketing/business type person, I set out to discover how to 'theme' my input forms." Breaks it into 4 steps: creating CCK content type -> create [yourcontenttype].tpl.php -> modify template.php -> modify style.css
• Content Templates module - CCK can generate ugly content; this module adds a "template" tab to CCK editing pages pre-populated with CCK's default layout, making it easy to customize how the fields are output. "But Content Template can actually be used on any node type and allows modification of the teaser and body properties before they go out in an RSS feed or are handed off to the theme."
• Converting 4.7.x themes to 5.x

Good to know

• Customization & theming handbook on drupal.org (extensive) - "Included in this section are PHP and SQL code snippets and examples for use in your sites pages, blocks and themes. There are also a few articles on theming engines, which provide the infrastructure to build and create new themes."
• Basic syntax for getting accessing the drupal CVS repositories
• Google Maps API documentation - if only all documentation was this good...
  
  

Features of iGEM2007.com

I've looked at a lot of drupal modules now and found much more seemingly-useful documentation. I can't even keep it all in my head at the moment, so I'm going to gather it all together in this post. Hopefully it will be easier to see how it all might fit together once its clustered on the same page. But first, I'm going to briefly describe the goals of the site again, the basic features, to help contextualize the following list of drupal modules & techniques.

Profiles
Users and the teams they comprise should have rich profiles that are easy for them to edit and easy for us to remix in new views.

Content Types
Users can create and revise a variety of content types: their profiles ( which will really be a synthesis of several different content types), posts to their team's blog, certain elements of their team's profile, their team's wiki pages, forum posts, comments, and eventually, media rich content types like embedded youtube / brightcove video and their history of promoting / digging all of the aformentioned types of content (perhaps not comments).

User Access Control
Users can only edit content that is associated with their team (i.e team blog, team project abstract) or associated with the entire site (i.e. comments, forums) - they shouldn't be able to edit another teams content.

Team Map
The team profile should contain fields that geolocate the team (either address or lat / lon). The site should feature a mashup of all the teams on a google map, presenting content like the team name, picture, abstract, and interesting recent activity like blog posts (& forum posts & promoted content).

RSS
RSS feeds should be available for all episodic content: team blogs, forums, and other interesting recent activity (diggs, new vodcasts).

User Registration & Team Application
Teams must satisfy several requirements before admitted into the competition:

• team leaders (2+ faculty members) must register all of their team members for user accounts (establishing the roster),
  
• submit the team proposal, which explains the general logistics of the team's project (where they will get wetlab space, reagents/materials, a place to meet as a team, funding for the entry fee, materials, trip to the jamboree, student stipends, etc.),
  
• send us the entry fee,
  
• and complete an initial portion of their team's profile focused on administrative information such as shipping address (and lat/lon if we can't geocode it), team name, official team name, affiliated institutions, etc.

The website must provide feedback to teams (on their profile page?) on their progress through the application process, making it obvious to them and us what steps have been completed and which are left.

Help Documentation
The new site should take advantage of the types of content built for writing documentation (like Book) and port over all the help documents from the 2006 igem wiki.

iGEM2007.com drupal dev notes, feb 07

TODO:

• Finish reading the IBM DeveloperWorks docs for drupal

• Follow along with the archived (currently 5) Drupal Dojo lessons/screencasts

• Start writing PHP via the docs at PHP.net and elsewhere

• Take a first pass at making a dynamic team map using the Google Maps API

I'll update this list as I learn.

OpenWetWare 2.0

TK pointed out in a conversation today that theoretically, what we really need for recording labwork digitally is something that provides the episodic, syndicated content of blogs with the collaborative and revision qualities of a wiki.

Much of the utility of such a system would come not just from having a record of your work online, but from the network effects engendered when that information was well-formed and tagged and shared in a quasi-standard way (the lower-case semantic web) with a community of similar users. The devil is in the details, and sharing information at such a fine level of granularity in a way that lets users pay attention to just what content is most likely to be relevant (via tags and perhaps some predictive metrics) could help users not make the same mistake collectively more than once. It might even allow users to perceive and draw conclusions about phenomena that were too subtle or rare to be investigated previously.

Help develop iGEM's drupal-powered site!

Hello, are you experienced with building sites based on drupal and/or knowledgeable of PHP and not averse to learning about drupal? If so, join us!

The International Genetically Engineered Machines Competition is now entering its third year and is expecting an increase in participation from 34 undergraduate teams from around the world to about 80-100. The teams spend the summer learning & doing genetic engineering and then present their work at the beginning of November here in Boston at MIT.

Last year we required all the teams to document their projects on a mediawiki server (http://parts.mit.edu/igem) - iGEM is not just an event that happens in November, although that's when it is most visible, but throughout the whole summer, and we wanted to capture as much of that experience online as possible. We built another website to explain the competition and link into each team's wiki page from a world map (http://igem2006.com).

I am developing a dynamic, community-driven site for the iGEM 2007 competition. I've spent the last two weeks investigating CMSs and laying out what features we want the new site to have, and I'm fairly sure Drupal is the base from which we should start. However, it's clear that we are going to need to develop a couple custom modules to support unusual features of the site. For instance, we will have about 1000 users, each a member of a certain team. We don't really want user accounts for anyone else. So we need to develop permissions for users based on which team they are in, and streamline how participants are given accounts ( i.e. manually authorizing 80-100 team leaders, who then can authorize their own team members). We also want each team to have its own portal page, dynamically generated from posts to the team's collective blog, recent forum activity, recent activity on our genetic parts database, and from certain pages from that team's space on our mediawiki.

I am trying to learn drupal and php as fast as I can to implement what we want, but I don't think it will be fast enough. Would you be interested in helping, or know anyone who might? How much would consultation / module development cost? If nothing else, pointers into the right direction would be very helpful.

If you are interested, I encourage you to check out our introductory site (I didn't design it!) at http://igem2006.com, and perhaps my development blog at http://cis-action.com.