PublishPlugin

Generates a static view of a web, as HTML files on disc, or as a PDF, or as a zip or tgz archive file, or by uploading directly to an FTP server.

Originally written for TWiki, previously known as GenHTMLAddOn, PublishAddOn, and PublishContrib, this is the most complete publishing solution for Foswiki.

PublishPlugin provides support for the generation of stand-alone HTML from a web. It will generate fully rendered versions of a set of Foswiki pages together with any attached files.

When you want to read a document stored in Foswiki, you have to have access to the web server that hosts that document. There are times when this may not be desirable, or even possible. For example:
  1. Foswiki is used to create documentation which has to be readable off-line
  2. Published versions of Foswiki pages must be read-only
  3. The Foswiki server is inaccessible to the audience (e.g. on the other side of a corporate firewall)
  4. You want to put product documentation on a CD
The PublishPlugin supports the generation of several different read-only document formats from Foswiki pages, including HTML and PDF.

Features

  • All standard Foswiki macros are interpreted, and plugins are called
  • Topic links internal to the wiki are translated to relative links
  • Powerful support for choosing what content gets published - fully compatible with BookmakerPlugin
  • Any links to the 'pub' areas of topics in the web are automatically resolved, and the referenced files copied
  • Any links to images outside the wiki are resolved, and the image is stored in the output (requires LWP)
  • Output in HTML or PDF. HTML can be compressed in different archive formats.
  • Full support for hierarchical webs
  • Multiple instances (e.g. dev, test, prod) can be specified
  • Special output format specific templates (such as viewpdf) can be used
  • Able to publish HTML and referenced files directly to a remote server via ftp
  • Complete history of what was published, and when!

Usage

Publish Form

The quickest way to publish a web is by completing the following form.

The output is generated in a directory designated during installation. The progress messages printed during documentation generation tell you exactly where the output is. Admins can use the PublishPluginControlCentre to manage the published output.

Publishing is an access-controlled process; before you can publish, you have to have VIEW access to the topics you want to publish, and CHANGE access to the publishing history topic.

You can also create a permanent topic in a web to help with a repeated publishing process.

HELP Most publishing tasks will only require setting the immediately visible options. Other more advanced options can be accessed by opening the collapsible sections.

Choose what to publish URL Parameter
Web The Web to publish. This is required to provide a context for the publishing process (e.g. web preference settings), even if all topics in the topiclist have explicit web names. web
Topic List Comma-separated list of topic names. Leave blank to publish all topics in the web (as filtered by inclusions and exclusions).
You can use Foswiki macros (such as %SEARCH) in this list - for example, to include the contents of a topic that contains a list of topic names in a bulleted list, use %SEARCH{"^   \*" type="regex" nonoise="on" topic="TheTopic" format="$pattern(.*?^\s*\*\s*(.*?)$.$)"}%.

You can also use the Bookmaker (see Using Bookmaker, below) to build such lists, and include them using the %BOOKLIST{"Bookweb.BookName"}% macro.

A non-blank topiclist is still subject to filtering against inclusions and exclusions.

Note that this list can include topics in other webs, by referring to them using the Web.TopicName syntax.
topiclist
Include filter Comma-separated list of wildcard patterns that match the names of topics to include. This acts as a filter on the topic list. Use * to publish all topics in the list. inclusions
Exclude filter Comma-separated list of wildcard patterns that match the names of topics to exclude. This acts as a filter on the topic list. Leave blank to include all topics in the topic list. exclusions
Versions Name of a topic in each published web that contains a table, each row of which maps topic names to the version of that topic to publish.
The table can be generated by a %SEARCH{}% or other macro. For example: |Web.TopicName|1.33|.
If a topic does not appear in the table, the most recent version will be published.
versions
Filter topic contents A regular expression that will cause a topic to be excluded if the RE matches the topic content. topicsearch
Processing options
Default processing of topics for publishing tries to render the topics as closely to the way they are viewed in the wiki as possible. These options provide a finer level of control.
Publish skin The skin provides the template for how topics are published. See Skins for more informations on skins. You are recommended to pick basic_publish, or plain, or a print skin. Your installation may also offer a special export or publish skin.
IDEA! The view template is used to generate published pages, so view.basic_publish.tmpl is the template that will be used to generate the output. You can preview any topic in this skin simply by appending ?skin=basic_publish to the end of the view URL. Note that the standard VIEW_TEMPLATE template override still works when publishing.
publishskin
Preferences Lets you define Foswiki preferences that will be available for use in topics during this publishing run. Define preferences one per line, using the syntax PREFERENCE=VALUE - for example,
TOOL_VERSION=3.14.15
ISBN=1-56592-149-6
Preferences defined this way can be used in topics (including the history topic like any other Foswiki preference.
ALERT! spaces after the = sign are taken as part of the preference value.
preferences
Enable plugins (plugin names)
Comma-separated list of Plugins to enable during publishing.

If you leave this field blank then all plugins enabled in the wiki will be run when generating the published output. You are recommended to disable any plugins that generate interactive buttons in the output.
enableplugins
Templates Comma-separated list of template names.

By default the plugin uses the default view templates when it renders topics for publishing. You can change this to a list of one or more templates that you want to render using.

For example, you might want to use viewpdf to render a PDF version of the topic (if you have this action available). See Skins for a list of the template files available by default.

See where to find the output for a complete description of where the output is generated.
templates
Copy External Resources Copy externally referenced resources (e.g. images on other servers). This option enables the copying of resources hosted on external servers into the published content. If it is disabled, the plugin will maintain an internet link to the external content. Enable this option if you want the pubished content to be totally self-contained (for example, for offline reading) or disable it for faster publishing process and smaller output. copyexternal
Output options
See where to find the output for details of where output is created on the server.
Output format The available formats are:
%PUBLISHING_GENERATORS{format="
$name
$help
" separator=""}%
format
Publishing history topic This is where the history of your publishing is stored. Each time you publish, this topic is re-written with the log of the publishing process. You have to have "change" access to this topic. You can specify a topic in another web using the standard Web.Topic syntax. history
File output options
File output options are relevant when output is being written to local files on disc.
Output file Specify the root of the target output file name. outfile
Google file Generates the "HTML verification file" needed to verify your site claim. See Google webmaster tools. googlefile
Default page (topic) Name of topic to used to generate default.htm, index.html defaultpage
Extended directory path Path of a directory relative to {PublishPlugin}{Dir} where you want the output generated. '/' means generate the output directly in {PublishPlugin}{Dir}. This allows you to keep different publishings of the same content separated.

See where to find the output for a complete description of where the output is generated.
relativedir
Resource Directory When publishing to HTML, attachments, icons, etc, are output to this directory. The directory name is relative to the root of your published site. rsrcdir
Relative URL The base URL that your published Foswiki topics will reside at, for instance to create different publishing instances like dev, test and public. Use / if you will publish to the root of your site. See also Google webmaster tools. relativeurl
FTP output options
FTP output options are only relevant if the output format is ftp.
Server address Set to blank to proof the output prior to uploading to your site. destinationftpserver
Server path Path to upload to on the FTP server. destinationftppath
Username FTP server user name. destinationftpusername
Password FTP server password. destinationftppassword
Fast publish Speed up the ftp publishing by only uploading modified files. This will store a (tiny) checksum (.md5) file on the server alongside each uploaded file which will be used to optimise future uploads. Recommended. fastupload
Other output generator options
Some output generators (e.g. pdf) support extra options.
Extras Some output generators support extra options (e.g. for pdf, you can add htmldoc command-line parameters here, such as --linkstyle underline). These options are passed directly to the output generator. extras

Wildcard Patterns

A wildcard is a special string that you can put into a filename so that it matches a whole range of files:
String What it does Example What the example matches
* Matches any string, including an empty string. *Cheese* Every topic with "Cheese" somewhere in the name (but not "cheese")
? Matches any single character. Example1? Example10 and Example 1X but not example1
[...] Matches any one of the enclosed characters. A pair of characters separated by a hyphen denotes a range expression; any character that sorts between those two characters, inclusive, using the current locale's collating sequence and character set, is matched. If the first character following the [ is a ^ then any character not enclosed is matched. A - may be matched by including it as the first or last character in the set. A ] may be matched by including it as the first character in the set.
Within [ and ], character classes can be specified using the syntax [:class:], where class is one of the following classes defined in the POSIX.2 standard: alnum, alpha, ascii, blank, cntrl, digit, graph, lower, print, punct, space, upper, word, xdigit. A character class matches any character belonging to that class. The word character class matches letters, digits, and the character _.
B[aeiou]g Bag, Bog, Big, Beg, Bug

Where to find the output

Output is generated in the directory specified by the {PublishPlugin}{Dir} configuration setting. Administrators can manage the contents of this directory from the browser using the %PUBLISHERS_CONTROL_CENTRE% macro (see PublishPluginControlCentre).

The full path to the output depends on the settings of relativedir and templates in use. The full (file) path is given by:

{PublishPlugin}{Dir} / relativedir / template / outfile

If relativedir is not set, then it will be omitted from the path. The template is only included in the path when the publishing template is not the view template.

If outfile is not set in the parameters it defaults to the name of the format being published. Most formats generate a single file with a unique extension that identifies the format e.g. .pdf. When publishing a format that generates multiple files (e.g. file) then the final path will refer to a directory.

ALERT! The rendered data can get pretty big, and the publishing process itself puts a heavy load on the server, especially when using compression on large webs.

Regular Expressions

A perl regular expression. You can use a simple string here, which will be matched exactly, or you can read up on perl regular expressions on the web.

How-tos

Publishing a single topic

Create a link that invokes the rest script and pass the current topic in param inclusions:
<a class='foswikiPopUp'
href='%SCRIPTURLPATH{"rest"}%/PublishPlugin/publish?%REVARG%;
web=%BASEWEB%;
inclusions=%BASETOPIC%;
format=file;
skin=myskin;
defaultpage=WebHome'
rel='nofollow'>
Publish this page
</a>
(added newlines for readability).

Now you can use this link in a skin adaption to put the link (or button) on all pages. Example of a template:

%TMPL:INCLUDE{view}%

%TMPL:DEF{"printable"}%<span><a class='foswikiPopUp'
href='%SCRIPTURLPATH{"rest"}%/PublishPlugin/publish?%REVARG%;
web=%BASEWEB%;
inclusions=%BASETOPIC%;
format=file;
skin=myskin;
defaultpage=WebHome'
rel='nofollow'>Publish this page</a></span>%TMPL:END%

Using Bookmaker

The Bookmaker allows you to select a number of topics by visiting them in turn and adding them to the book. Once your book is complete, you can return to this page to publish it.

To start the bookmaker, enable the BookmakerPlugin, and revisit this page to enable the interface. Once the bookmaker is running, visit the pages you want to add to the book and add them to the book. Once you are finished, use the bookmaker interface to return to this page to publish the results. To publish a book generated by Bookmaker, use the %BOOKLIST{"Bookweb.BookName"}% macro in the topiclist parameter.

Using a Publish Topic (configtopic)

You can create a publish topic in a web that contains all the details needed to publish that web. This is just a topic with a series of standard preference settings (which correspond to the form parameters) in it. You can use the PublishWeb topic in this web as a template for your own topics.

Alternatively you can just take a copy of the form in this topic, paste it into your own topic, and change the defaults.

To use a publish topic, you must pass the configtopic parameter to the publish script set to the name of the topic to use to control publishing. You can specify a topic in another web using the standard Web.Topic syntax.

Publishing from the command line

Just cd to the bin directory, and perl -T rest /PublishPlugin/publish. Parameters are passed as name=value pairs, for example:
perl -T rest /PublishPlugin/publish web=Book exclusions='Web*' format=file
perl -T rest /PublishPlugin/publish web=Book inclusions=WebBook format=pdf extras='--book --duplex --toclevels=5'
The available parameter names are shown in the publish form above, in the last column. Where parameters take multiple values (e.g. preferences) then you can separate the values using semicolons.

Controlling which parts of a topic get published

You can control what gets published from a topic using %STARTPUBLISH% and %STOPPUBLISH% control tags:
  • If %STARTPUBLISH% is the first control tag seen in the file, everything before it will be ignored.
  • Everything between %STOPPUBLISH% and the next %STARTPUBLISH% (or the end of the topic) will be ignored.
  • %STARTPUBLISH% and %STOPPUBLISH% will be visible in the viewed topic, so you can easily see what will be published from the topic.
Note: the old <nopublish> tag is deprecated and should be replaced in topics

Another good trick is to set up a special "publishing" web. Create topics in the web that %INCLUDE the topics from other webs that you want to publish. You can use STARTSECTION and ENDSECTION to highlight what you want published. This way the "publishing" web gives you a view of exactly what will be in the published output, without the need for special publishing tags.

Publishing History

Every time a web is published, then the results of that publishing step are stored in a topic in the web. By default this topic is called PublishPluginHistory, but you can choose another name (see the form, above). In order to publish a web, you have to be able to write to this topic.

If the selected publishing skin defines a skin template called publish_history, then that template will be used as the basis of the history topic. This (for example) allows you to use a template with a skin to define access controls for the history topic. The template can refer to a Foswiki macro %PUBLISHING_HISTORY% to get the expanded history. The basic_publish skin provides templates/publish_history.basic_publish.tmpl for this purpose.

The history topic contains a list of all the parameters used, and the versions of the topics that were published, so it is very useful for tracking exactly what you publish. It is written every time you run publish.

Installation Instructions

Dependencies

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

IMPORTANT Run configure and complete the installation in the PublishPlugin section.

If you want to generate PDF files, you will need to install a PDF generator, for example htmldoc or prince. Find them using google.

Note that htmldoc can also be used to generate PostScript by using the -t option in the Other output generator options above. See the htmldoc man pages for details.

If you want .zip output you will have to install Archive::Zip and everything it depends on.

If you want .tgz (tar) output, install Archive::Tar and everything it depends on. Install Archive::Zip and everything it depends on

WARNING! Anything published is no longer under the control of Foswiki access controls, and if you make the publish output directory visible on the web then you may need to take precautions to prevent accidental leakage of confidential information by restricting web access to this directory, for example in the Apache configuration.

Info

This add-on started life as the GenHTMLAddon, written by Foswiki:Main/CrawfordCurrie at Motorola. It was then extended by Eric Scouten, and then further fixed and enhanced by Foswiki:Main/CrawfordCurrie (http://c-dot.co.uk). It has also been extended by Foswiki:Main/SvenDowideit and Foswiki:Main/MartinCleaver, and most recently refactored for Foswiki by Foswiki:Main/CrawfordCurrie. Other significant contributors are Foswiki:Main.ArthurClemens and Foswiki:Main.MichaelDaum.

Author: Foswiki:Main/CrawfordCurrie
Dependencies:
NameVersionDescription
File::Spec>0Required. Used to analyse URL paths.
File::Copy>0Required. Used to move files around.
File::Path>0Required to manipulate directories.
File::Temp>0Required for temporary files.
LWP>0Optional. Used to include images referenced by absolute URLs
Archive::Zip>=0Optional. Required to generate .zip output
Archive::Tar>=0Optional. Required to generate .tgz output
Net::FTP>0Optional. Required for ftp publishing.
htmldocOptional. Required to generate .pdf output
Digest::MD5>0Optional. Required for fast upload to ftp servers.
License: GPL
Version: 15237 (2012-07-31)
Release: 2.4.0
Change History:  
2.4.0 (30 Jul 2012) Foswiki:Tasks/Item12016 Add capability to construct flat HTML and PDF files. Bugfixes: Foswiki:Tasks/Item11988 Foswiki:Tasks/Item11339 Foswiki:Tasks/Item11345 Foswiki:Tasks/Item8260 Foswiki:Tasks/Item10597 Foswiki:Tasks/Item11346
2.3.2 (10 Aug 2011) Foswiki:Tasks/Item10944: Fix publishing of attachments
2.3.1 (14 Jun 2011) Foswiki:Tasks/Item10870: support skinning of the history topic. Foswiki:Tasks/Item10870: support definition of session preferences from the publish form. Foswiki:Tasks/Item10843: bugfix to publishers control centre
2.2.1 (25 May 2011) Foswiki:Tasks/Item10578: allow list of publishskins; Foswiki:Tasks/Item10580: allow empty outfile param; Foswiki:Tasks/Item10585: support all valid topic names in topiclist; Foswiki:Tasks/Item10578: made template purging configurable Foswiki:Tasks/Item10594: merged patch; Foswiki:Tasks/Item10581: correct paths to resources in HTML output. (Diab Jerius and Crawford Currie)
2.2.0 (28 Mar 2011) Foswiki:Tasks/Item8225: fix handling of plugin contexts Item2635: make sure index.html is generated Foswiki:Tasks/Item10529: test integration with BookmakerPlugin via new topiclist parameter
2.1.7 (01 Nov 2010) Foswikitask:Item8658: fix rest output when publishing to file, pdf and ftp
2.1.6 (29 Oct 2010) Foswikitask:Item8522: support for Foswiki 1.1.x and also 1.0.x+ZonePlugin Foswikitask:Item1638: fixed finding resources with parameters (e.g. ?t=2365421)
2.1.5 (05 Feb 2010) Documentation update.
2.1.4 (12 Jan 2010) Foswikitask:Item2557: fixed publish head elements added by ADDTOHEAD
2.1.3 (11 Jan 2010) Foswikitask:Item2615: fixed finding resources not inside quotes
2.1.2 (30 May 2009) Foswikitask:Item8168: fixed genopt (extras)
2.1.1 (22 May 2009) Foswikitask:Item8165: fixed missing BASEWEB and other internal preferences. This was resulting in the wrong web being used for some operations
2.1.0 (16 May 2009) Foswikitask:Item1626: fixed META{"formfield" Foswikitask:Item8150: (Marc Schaefer) fix for -skin parameter Foswikitask:Item1557: doc fix Foswikitask:Item1585: allow history topic in different web Foswikitask:Item871: add missing newline at start of topic text. Foswikitask:Item1449: topic publisher was not popping the context correctly Foswikitask:Item1632: improve backporting support
2.0.2 (18 Mar 2009) Foswikitask:Item804: automatically create publish dir Foswikitask:Item8078: support publishing to a subdir of the publish dir, under url param control
2.0.1 (14 Feb 2009) Foswikitask:Item1033: fixed button in PublishWeb
2.0.0 (27 Nov 2008) Foswikitask:Item8019: refactored as a plugin and tested in Foswiki
1.6.5 (27 Oct 2008) Item5385: Fixed doc for configtopic Item5388: $WEB and $TOPIC were not correct in %IF statements Item5390: remove comments from .css before processing for included resoures Item5706: Improved FTP upload process for incrementally maintained webs Item6029: expand config topic on load to support use of searches Item6030: respect VIEW_TEMPLATE in published topics Item6092: expand common tags in configtopic Item6094: move sitemap options into base file generator Item6110: rename settings in config topic to avoid clashes with other plugins
1.6.4 (11 Dec 2007) Item5099 fixed
1.6.3 (10 Nov 2007) Tested on 4.2.0. Item4624:, Item4625: Item4830: fixed. Item4825: added a basic skin to avoid the confusion caused by text skin. Item4951: added interface to allow management of output files
1.6.2 fixed ftp publish, added doco, and added enabled plugin selection funcitonality
1.6.1 Item3722 worked around core attaching URL params to internal URLs
1.6.0 Item3671 cannot publish without write access to history topic, so security now checked early. Item3619 Cleaned up error handling from writers. Item3675 added history topic to record changeset. Plus major refactoring of main class to get rid of some of the cruft that had built up from many authors. Item2726: uses getExternalResource now, so should obey proxy settings (untested)
1.5.5 Added support for new internal api - no user changes
1.5.4 Added UI for FTP. Added .spec file. Fixed Item3515 and Item2725
1.5.3 Michael Daum - create a new wiki object for every topic, don't reuse the current one (Item3139)
1.5.2 Correction to the correction for anchors.
1.5.1 Correction to support anchors in URLs properly
1.5.0 Martin Cleaver - changes to allow generation of viewprint and viewxxx when specified by TEMPLATE; multiple INSTANCE (dev/test/prod); (Item2269)
1.4.3 Bugfix Item2216
1.4.2 Crawford Currie - fixed problem where it was failing to remove <base> tags completely (Item2200)
1.4.1 Crawford Currie - added doc on usage from command line, corrected sense of topicsearch filter (Item2120, Item2121), renamed parameters (old ones are still valid), corrected handling of empty web refs (Item2128), deprecated nopublish html-style tag in favour of PublishWebPlugin-compatible style (though with richer semantics) (Item2196)
1.4.0 Crawford Currie - added support for hierarchical webs, and inclusion of external images.
1.3.9 Crawford Currie - added tgz and pdf support
1.3.8 Michael Daum - fixed rewriting urls; fixed nested resources issue; creating a new prefs object for each topic
1.3.7 Corrected form action so it uses up the right web preferences
1.3.6 Michael Daum - recursively archive resources imported by css files; fixed several html errors in the PublishPlugin and PublishWeb topics; removed hardcoded reference to print.pattern
1.3.5 TWiki-4 version. Also supports publishing to a file area, making TWiki easier to use as a CMS (see also Foswiki:Extensions/PublishWebPlugin, which does almost the same thing frown, sad smile )
1.3.4 Item196 - bugfix for HTTP_HOST, as described in the Dev topic for the contrib
1.3.3 Changed interface to support wildcards, and lightened the plugin by replacing a lot of files with simpler ways of doing things.
1.3.2 Added Compress::Zlib dependency, as requested by Brad Taylor
1.3.1 (27 Apr 2005) Crawford Currie - fixed minor issues highlighted by Bruce Dillahunty and Scott Claridge
1.3.0 (11 Apr 2005) Crawford Currie - reworked the interface and code to work better
1.2.0 (13 October 2004) Crawford Currie - Cairo compatible
1.1.0 (7 Jan 2003) Initial version
Home: http://foswiki.org/Extensions/PublishPlugin
Support: http://foswiki.org/Support/PublishPlugin

Related Topics: DefaultPreferences, SitePreferences, Plugins

This code is a development of the Architectures and System Platforms group of Motorola Inc. and is protected by the following copyrights:
  • Copyright © 2001 Motorola. All Rights Reserved.
  • Copyright © 2002-2003, Eric Scouten.
  • Copyright © 2004-2006 Crawford Currie http://c-dot.co.uk
  • Copyright © 2006 Martin Cleaver http://www.cleaver.org

The 2005 functionality improvements were sponsored by Wind River Systems

The pdf and tgz output formats were made possible by Sabio Labs

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details, published at http://www.gnu.org/copyleft/gpl.html
Topic revision: r1 - 31 Jul 2012, UnknownUser
This site is powered by FoswikiCopyright © by the contributing authors. All material on this site is the property of the contributing authors.
Ideas, requests, problems regarding GSICS Wiki? Send feedback