Site Statistics

Introduction

Statistics of visits to pages in a web can be generated manually or automatically, on a per-web basis. Statistics are compiled as a running total for each month.They include totals for Topic Views, Topic Saves, Attachment Uploads, Most Popular Topics with number of views, and Top Contributors showing total of saves and attachment uploads. Statistics from previous months are saved, and a new row is written to the table at the beginning of each month.

• You can create a WebStatistics link using %STATISTICSTOPIC%

Statistics are compiled by a script that processes the Foswiki event log. The script can be run two ways:

• Manually, by an authorized user from a browser URL
• Automatically by a shell script, usually run as a cron job

Manually running statistics

The following form can be used to run statistics and create the WebStatistics topics for selected or all webs.

Note: The current user - WikiGuest must have access rights to read/write the WebStatistics topics, and if missing, must have authority to create the WebStatistics in the selected webs.

Run statistics on a list of one or more webs

Enter a comma separated list of webnames. Enter a subweb by specifying ParentWeb/SubWeb/ Web names must follow web naming rules and start with an upper case letter.

Additional Settings

Autocreate missing WebStatistics

Select this option if you want the Statistics script to auto-create the WebStatistics if it is missing. (Note: this operation must be enabled in the configuration).

Process sub-webs

Process any subwebs of the requested webs. If this is not selected, then sub-webs must be explicitly listed in the list of webs.

Specify date to process

Specify the date of the logs to process in YYYYMM format. Example: 201203 to request March 2012.

Controlling access to the statistics script

By default, the statistics script may be run by anyone, however the user must have CHANGE permission on the WebStatistics topic. Statistics generation can create a significant workload on the server. In order to further restrict statistics generation:

• Add statistics to the configure Security and Authentication tab, Login sub-tab.(This is now the default on Foswiki 1.1.4). This is required to prevent unauthenticated guest users from running statistics.
  • If using Apache authentication, be sure to add it to the Apache configuration.
• If desired, create a Group for users that should be permitted to run the statistics script, or use the AdminGroup.
• Visit configure Logging and Statistics tab, and set {Stats}{StatisticsGroup} to the group permitted to run statistics.
  • Note: Regardless of this setting, members of the AdminGroup can always run statistics.

Configuring for automatic operation

You can automatically generate usage statistics for all webs. To enable this:

• By default, the WebStatistics topic must be present in all webs where you want to have statistics. See below for how to create the WebStatistics topic.
• Call the bin/statistics script from a cron job, once a day is recommended. This will update the WebStatistics topics in all top level webs. (Sub-webs are not processed by default).
• Caution: The script must run as the same user as the Web Server CGI scripts. This varies by platform, but is typically apache, www. or nobody on many systems. Verify this before running statistics from the command line or cron. If the ownership of critical files is changed, it can render the Foswiki server inoperable.
• Example crontab entry to process all webs and subwebs containing a WebStatistics topic:
  0 0 * * * (cd /path/to/bin; ./statistics -subwebs 1 >/dev/null 2>&1)
  • This crontab entry should be installed for the web server user, not root!

.

When running from the command line or a cron job, you can pass parameters to the script like this:

./statistics -logdate 200605 -webs Userweb,Sandbox -autocreate 1 -subwebs 1

Parameter	Description	Default
autocreate	Flag to request auto-creation of missing WebStatistics topics. 0=false 1=true	(See logging and statistics page in bin/configure.)
logdate	Generate statistics for the specified year/month, spacified as YYYYMM	current month
subwebs	Flag to request processing of subwebs of the requested webs. 0=false 1=true	0 (Subwebs are not processed)
webs	comma-separated list of webs.	all accessible webs

Creating the WebStatistics topics

As of release 1.1.4 of Foswiki, the WebStatistics topics are no longer shipped by default. The bin/statistics script can automatically create the required topics when enabled in the configuration, or when requested by URL Parameter autocreate=1.

• If you want the Statistics topic to be automatically created whenever needed, in every web
  • Visit http://gsics.atmos.umd.edu/bin/configure Logging and Statistics tab, and set {Stats}{AutoCreateTopic} to Always

• If you want to manually add the WebStatistics topic to a subset of webs
  • Visit http://gsics.atmos.umd.edu/bin/configure Logging and Statistics tab, and set {Stats}{AutoCreateTopic} to Permitted
  • Use the below form to run statistics on one or more webs.
  • or add -autocreate 1 to the statistics script as shown above.

• If yow want to prohibit autocreate of WebStatistics topics
  • Visit http://gsics.atmos.umd.edu/bin/configure Logging and Statistics and set {Stats}{AutoCreateTopic} to Prohibited
  • Note: This is the default behavior.

Generating statistics manually by URL

The bin/statistics script can also be executed as a CGI script, Use the form at the top of this topic to run statistics.

Tailoring the WebStatistics layout

The WebStatistics topics are not shipped with Foswiki by default. The topics are created on demand by the statistics script when requested by configuration. The topics are created as follows:

1. The Main web is searched for WebStatisticsTemplate - if this topic does not exist
   • The System web is searched for WebStatisticsTemplate. (This topic is shipped with Foswiki by default)
2. If {Stats}{AutoCreateTopic} is enabled in the configuration, or the autocreate parameter is passed to the statistics script, then a new WebStatistics is created using the WebStatisticsTemplate topic found in the first step.
   • If autocreate is not requested, and the WebStatistics topic does not exist, then statistics will not be computed for the web.

The template topic contains an %INCLUDE% for help text and table headings, and a critical marker line that defines the columns to be recorded by the statistics script. (This allows the help text or headings to be modified without needing to edit every WebStatistics topic.)

• The WebStatistics topic looks for DefaultWebStatistics - if this topic does not exist.
  • The DefaultWebStatistics is included. (This topic is shipped with Foswiki by default).

To tailor the help text or column heading, copy DefaultWebStatistics to DefaultWebStatistics and tailor as desired. This prevents your changes from being lost when Foswiki is updated.

To tailor the actual columns recorded and the order of the columns, copy WebStatisticsTemplate to WebStatisticsTemplate and tailor as desired. Then remove the previous WebStatistics and allow the statistics script to auto-create a new topic based upon the new template. Alternatively you can edit the WebStatistics in each web.

Statistics are written into the WebStatistics in the following order.

• If a marker line is found in the topic, statistics are inserted into the topic after the marker line.
• If no marker line is present, but the topic contains existing statistics, the current month is update, or a new month is written after the previous month.
• If no marker line is present and no previous statistics exist, a new line is added to the end of the topic.

Removal of the marker line causes statistics to be gathered in chronological order (oldest first). By default, statistics are reported in reverse chronological order.

The marker line defines the order of the columns collected by the statistics script. The order of the columns should correspond to the table heading in the DefaultWebStatistics. Default marker line is:

| <!--statDate--> | <!--statViews--> | <!--statSaves--> | <!--statUploads--> | <!--statTopViews--> | <!--statTopContributors--> |

Restricting access to statistics topics

WebStatistics topics can have access controls like other topics.

If you want to have these access settings copied to new WebStatistics topics, follow these steps:

1. Copy topic WebStatisticsTemplate from System web to Main web
2. Edit the topic settings (via the "More" link) and write (if you want to restrict topics views to logged in users):

         * Set DENYTOPICVIEW = WikiGuest
   

For existing WebStatistics topics you will need to write these settings manually.

Managing the WebStatistics topics

Each WebStatistics topic will continue to grow, a new row added every month. In addition, every update of the statistics topic will create a new entry in the WebStatistics topic revision history. These can become extremely large over time and Foswiki does not provide any mechanism for automatically archiving statistics topics. For good ongoing performance, it is recommended to archive statistics annually:

• Enable {Stats}{AutoCreateTopic} or add -autocreate 1 to the cron job that runs the statistics script.
• Rename the WebStatistics to WebStatisticsYYYY at year end.
• Optional: Tailor the DefaultWebStatistics topic, adding a search for archived statistics, for example:

%SEARCH{"." type="regex" topic="%STATISTICSTOPIC%*" excludetopic="%BASETOPIC%" scope="topic" nonoise="on" header="*Other Statistics Topics:*" format="   * $web.$topic"}%

Following these steps will start a fresh statistics topic with a clean revision history.