Tag Archives: Plone

Create and Download a CSV (or any other file type) with a Plone View

Do you have a content type that you’d like to be able to offer as Comma Separated Value (CSV)/Excel download? Python can create both formats pretty easily (using either the standard csv library, or xlwt) and you can write a view for your content type that will create the file and return it to the user as a file download.

At Plone.org, Martin Aspeli covers this process as a part of the five.grok documentation. I’d already created my product without using five.grok, and I didn’t figure this one use case was enough reason to switch. So here’s what I came up with:

First, register the view in browser/configure.zcml

...
    <browser:view
      for="my.product.interfaces.IMyContentType"
      name="csv_view"
      class=".mycontenttypeview.MyContentTypeCsvView"
      permission="zope.Public"
      />
...

I added the view to browser/myContentType.py under the default view.

...
class MyContentTypeCsvView(BrowserView):
    """
    Download the content type as a CSV file.
    """
    
    def __call__(self):
        """
	Build a CSV from the content type.
	"""
	out = StringIO()
	writer = csv.writer(out)

        ...
        # make the CSV file
        ...

	filename = "%s.csv" % context.id

	self.request.response.setHeader('Content-Type', 'text/csv')
	self.request.response.setHeader('Content-Disposition', 'attachment; filename="%s"' % filename)

        return out.getvalue()

Now you should be able to link to http://mysite.com/mycontenttype/csv_view and initiate a file download.

How to view the contents of the browser in ZopeTestCase

In first writing tests for Plone products, it was difficult for me to figure out what was in the browser when a failure occurred. Adding the two following lines just after the offending code will create a file ‘test.html’ that you can view in a real broswer:

>>> f = open('/Users/kevin/Desktop/test.html','w')
>>> f.write(browser.contents)

Change the path as necessary to point to your desktop, run the test, and click the file on your Desktop to figure out what the heck is going on.

Adding a new user in a ZopeTestCase doctest

One of the content types in a Plone product I’m working on needed to refer to one of the users in the site. Therefore I needed to get some users in the system in the doctest. At first I tried .click()ing through the interface to get to add a new user through Site Setup. After much wailing and gnashing of teeth, I realized I could do this in straight Python:

>>> pr = self.portal.portal_registration
>>> pr.addMember(id = 'ddastardly',
...              password = 'password',
...              roles = ["Member",],
...              properties = {'fullname': 'Dick Dastardly',
...                            'username': 'ddastardly',
...                            'email': 'ddastardly@example.com',}
...              )

This is cool because I don’t need to test the Plone “Add a New User” interface as a part of my product. I only need a few users available to test my content type. Later, if you need to do anything as this user, you can log in like this.

>>> browser.getLink('Log in').click()
>>> browser.getControl(name='__ac_name').value = 'swhiplash'
>>> browser.getControl(name='__ac_password').value = 'password'
>>> browser.getControl(name='submit').click()
>>> 'You are now logged in' in browser.contents
True

Accessing radio buttons & checkboxes in the ZopeTestCase browser

This definitely falls into the ‘reminding-myself-for-next-time’ blog category, but maybe this will help someone else out. I found myself needing to test radio buttons that didn’t have labels (don’t ask why). Those radio buttons all shared the same name attribute. Trying to access them by the name got the larger ListControl, not any of the radio buttons themselves:

>>> # here the radio buttons have the name "age_group"
>>> from Products.Five.testbrowser import Browser
>>> browser = Browser()
>>> browser.getControl(name='age_group').click()
>>> .... (continued)
>>> AttributeError: 'ListControl' object has no attribute 'click'

You can, however, access the individual radio buttons from the ListControl using the controls attribute on the ListControl. controls returns a list of the ListControl’s child controls, which you can access by index.

>>> # here the second radio button's value attribute is '20-25'
>>> browser.getControl(name='age_group').controls[1].selected = True
>>> browser.getControl(name='age_group').value
['20-25']

Benchmarking Plone 3.3.5 with ApacheBench

Ever wondered what a good caching policy can do for your site’s performance? I’m deep in the throws of a ground-up Plone installation behind Apache, and I’ll be using CacheFu and Varnish to speed up the site. As I go along I’ll run ApacheBench at major milestones to see what difference they’re making.

The Command

ApacheBench comes installed with Apache, so no need to add anything new. To benchmark my site, I’ll run the following command:

ab -n 100 -c 3 http://127.0.0.1:8080/Plone/front-page

That’s 100 total requests, no more than 3 at the same time.

Initial Data

Here are the results from the “Welcome to Plone” page on a freshly created Plone site.

Server Software:        Zope/(unreleased
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /Plone/front-page
Document Length:        21120 bytes

Concurrency Level:      3
Time taken for tests:   34.529235 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      2140200 bytes
HTML transferred:       2112000 bytes
Requests per second:    2.90 [#/sec] (mean)
Time per request:       1035.877 [ms] (mean)
Time per request:       345.292 [ms] (mean, across all concurrent requests)
Transfer rate:          60.53 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   4.4      0      32
Processing:   263 1021 3556.2    528   34526
Waiting:      261 1019 3553.4    523   34494
Total:        263 1022 3557.4    528   34526

Percentage of the requests served within a certain time (ms)
50%    528
66%    553
75%    564
80%    575
90%    667
95%    967
98%   8281
99%  34526
100% 34526 (longest request)

Next, let’s see what happens when we add content.

With Data, Running on the Bare Zope Server

Since the site is running on ZEO, I decided to temporarily leave one client running on the bare Zope application server and set up one behind Apache. Here’re the stats for the non-Apache client.

Server Software:        Zope/(unreleased
Server Hostname:        127.0.0.1
Server Port:            8081

Document Path:          /Plone/
Document Length:        26136 bytes

Concurrency Level:      3
Time taken for tests:   42.539567 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      2641800 bytes
HTML transferred:       2613600 bytes
Requests per second:    2.35 [#/sec] (mean)
Time per request:       1276.187 [ms] (mean)
Time per request:       425.396 [ms] (mean, across all concurrent requests)
Transfer rate:          60.63 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   15 115.0      0     991
Processing:   408 1247 221.9   1221    2181
Waiting:       10 1235 253.2   1219    2179
Total:        820 1263 189.2   1221    2181

Percentage of the requests served within a certain time (ms)
 50%   1221
 66%   1225
 75%   1227
 80%   1231
 90%   1409
 95%   1596
 98%   2181
 99%   2181
 100%   2181 (longest request)

With Data, Running Behind Apache

… and here’s what I got on the client running behind Apache. I hadn’t spent any time on caching yet.

Server Software:        Zope/(unreleased
Server Hostname:        **************************
Server Port:            80

Document Path:          /
Document Length:        25745 bytes

Concurrency Level:      3
Time taken for tests:   46.875144 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      2602800 bytes
HTML transferred:       2574500 bytes
Requests per second:    2.13 [#/sec] (mean)
Time per request:       1406.254 [ms] (mean)
Time per request:       468.751 [ms] (mean, across all concurrent requests)
Transfer rate:          54.21 [Kbytes/sec] received

Connection Times (ms)
 min  mean[+/-sd] median   max
Connect:        0   12  87.6      0     740
Processing:   719 1385 2949.9    899   24272
Waiting:      264 1377 2935.4    899   24271
Total:        719 1397 2978.6    901   24272

Percentage of the requests served within a certain time (ms)
 50%    901
 66%    934
 75%    988
 80%   1031
 90%   1453
 95%   1954
 98%  19548
 99%  24272
 100%  24272 (longest request)

With Data, from my local machine

This is what it looks like from my local machine.

Server Software:        Apache/2.2.3
Server Hostname:        ***********************
Server Port:            80

Document Path:          /
Document Length:        0 bytes

Concurrency Level:      3
Time taken for tests:   3.004 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Non-2xx responses:      100
Total transferred:      25500 bytes
HTML transferred:       0 bytes
Requests per second:    33.29 [#/sec] (mean)
Time per request:       90.124 [ms] (mean)
Time per request:       30.041 [ms] (mean, across all concurrent requests)
Transfer rate:          8.29 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       26   44  13.6     39     101
Processing:    27   45  14.8     40     100
Waiting:       26   45  14.9     40     100
Total:         54   89  22.1     85     161

Percentage of the requests served within a certain time (ms)
  50%     85
  66%     95
  75%    101
  80%    108
  90%    122
  95%    137
  98%    143
  99%    161
 100%    161 (longest request)

In GenericSetup, a Page isn’t a “Page”

Here’s a little gotcha I just ran into again recently. I’ve got a new folderish Plone content type that I’m working on, to which I’d like to be able to add plain-old Pages. At first, my profiles/default/types/NewFolderish.xml looked like this:

...
<property name="filter_content_types">True</property>
<property name="allowed_content_types">
  <element value="Link" />
  <element value="Page" /> <!-- WRONG -->
</property>
...

Well, that didn’t allow me to add a page to a New Folderish content type.  Here’s why: for GenericSetup, you have to identify a content type by it’s meta_type. For most default content types, it’s what you’d expect. For Page and Collection, it’s not. Here’s the correct profiles/default/types/NewFolderish.xml

...
<property name="filter_content_types">True</property>
<property name="allowed_content_types">
  <element value="Link" />
  <element value="Document" /> <!-- CORRECT -->
</property>
...

To allow a Page

  <element value="Document" />

To allow a Collection

  <element value="Topic" />

Happy Ploning!

Plone Buildout on Snow Leopard… from the ground up

This is how I got a fresh Plone buildout up and running on my brand new iMac. I’m writing this mostly for my designer buddies here in Nashville, in the hopes that one day they will come to the light and start skinning Plone sites with me. We can dream can’t we?

Major thanks to Brian Gershon (without this blog post it would have taken me until the Ides of March to get through this), and especially Florian Schulze for his buildout that makes Python 2.4 on Snow Leopard a snap.

Step 1: Find Your Mac OS X Install DVD

Okay, if you’re like me, you don’t know where your install DVD is. Unless, of course, you’ve broken the seal on your new Mac less that 24 hours prior.

If you have your Install DVD:

  1. Insert the DVD, open it, and open “Optional Installs”
  2. Double click “Xcode.mpkg”
  3. Let Xcode install on your machine. It took me between 15 and 20 minutes.

If you DON’T have your Install DVD:

  1. Go to http://developer.apple.com/TOOLS/Xcode/
  2. Click “Download Latest Xcode”
  3. Create an ADC membership if you haven’t already (it’s free)
  4. Download the package (it’s a doozy, 2 GB+)
  5. Double click the package file after you’ve downloaded it
  6. Let Xcode install

Once you’ve gone through the install process, open up Terminal by going to Applications -> Utilities (go ahead and drag it to the dock, you’ll need it later). Type ‘gcc’ into Terminal and press Return. You should get something like the following:

i686-apple-darwin10-gcc-4.2.1: no input files

What’s that, you say? Why, that’s your brand new GNU C Compiler telling you it needs an input file before it can do anything! If you see something different, you probably had a problem installing Xcode.

Step 2: Installing Python 2.4 (and other goodies)

As I’m still working in Plone 3, I’ll have to install Python 2.4. SL ships with Pythons 2.5 and 2.6. Usually it’d be easy to install different versions of Python, but according to Brian’s post SL mangles up a Python 2.4 installation. Here come’s Florian’s buildout to the rescue.

NOTE: This installation method is specific to some problems that Snow Leopard had with Python 2.4. Details available here and on Brian’s blog.

To Install Python 2.4 and the rest of the gang:

  1. Open Terminal
  2. Create a new directory called ‘src’ in your home directory by typing in the following command
  3. mkdir src
  4. Move into that directory with this command
  5. cd src
  6. Checkout and run Florian’s buildout with the following commands
  7. svn co http://svn.plone.org/svn/collective/buildout/python/
    cd python
    python bootstrap.py
    bin/buildout

You should now be able to run Python 2.4 by typing the following code into Terminal:

python-2.4/bin/python2.4

That’s the Python2.4 deep inside your user directory. As it stands, you’d have to use the full path to that file to run Python 2.4. Let’s change that.

Symlinking Python 2.4 to the path:

The goal here is to be able to run Python 2.4 by simply typing “python2.4” into the Terminal. To accomplish this:

  1. Change to a directory in your path (I used /usr/bin)
  2. cd /usr/bin
  3. Type in the following command, substituting your username for “kevin”. You’ll have to type in your password, as we’re editing system files here (type carefully!)
  4. sudo ln -s /Users/kevin/src/python/python-2.4/bin/python2.4 python2.4
  5. Go back to your home folder, and try running Python 2.4 without the path
  6. cd ~
    python2.4

If you get a Python 2.4 interpreter (see below), you’re golden.

Python 2.4.6 (#1, Dec 29 2009, 23:33:05)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

While we’re at it, let’s symlink easy_install-2.4 as well. We’ll need this for ZopeSkel:

cd /usr/bin
sudo ln -s /Users/kevin/src/python/python-2.4/bin/easy_install-2.4 easy_install-2.4

If you ever happen to meet Florian Schulze, thank him for this buildout.

Step 3: Creating a Plone 3 Buildout (finally, right?)

To make a buildout, we’re gonna need Paster. To get Paster, we’re gonna need ZopeSkel. Let’s use that newly symlinked easy_install-2.4 to get ’em.

cd ~
easy_install-2.4 -U ZopeSkel

This installs what we need, but again deep in our user directory. A symlinking we will go:

cd /usr/bin
sudo ln -s /Users/kevin/src/python/python-2.4/bin/paster paster-2.4

Notice I named this particular Paster “paster-2.4”. That’s in case we ever install paster for a different version of Python, we’ll know which one is which.

Now let’s pick a home for our buildouts. I like to create a directory called “workspace” next to my “src” directory (sigh… good ol’ Eclipse). Make that directory, and move into it with these commands:

mkdir workspace
cd workspace

To start a new buildout, type the following command:

paster-2.4 create -t plone3_buildout myplone

Paster will ask you a lot of questions, like what version of Plone you want, and which Zope to use. Hit enter to accept all of the defaults. When Paster is done running, go into the directory you just created, bootstrap the buildout, and run buildout with the following commands:

cd myplone
python2.4 bootstrap.py
bin/buildout

If buildout ran without any errors, the last line of the Terminal output will look like this:

Generated interpreter '/Users/kevin/workspace/myplone/bin/zopepy'.

Step 4: Starting Plone

With our buildout created, now we simply start Zope with the following command:

bin/instance start

Point your browser to http://localhost:8080/manage_main, type in the username and password you supplied to Paster, and add a Plone site through the ZMI. Welcome to Plone on Snow Leopard!

E-mail notification for scripted Plone users

Here’s a little something that I googled and googled for but never found:

I was using Andy McKay’s script for adding users in bulk from The Definitive Guide to Plone (1st edition). It’s a great little script, but I needed to send the newly created users an e-mail to notify them of the new account. Long story short, the key is the registeredNotify() method available on the portal_registration tool:

def importUsers(self):
 reader = csv.reader(open(fileName, "r"))
 pr = self.portal_registration

...

# send e-mail notifying user
 pr.registeredNotify(id)

That’s it. registerNotify() is the same method that {portal_url}/join_form uses to send an e-mail with a link to create a new password. Just add those two lines into the script and you’re cookin’.