Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

TL;DR: How do I add a text (notice) to the content (not a header or global template) of every page?

(very) long question with background: I'm planning to migrate a MediaWiki over to another wiki. The content of the wiki has been migrated from an even older wiki before (where errors regarding formatting were generated), grown over time and is now outdated in large parts. That's why we want to start with a blank wiki and migrate the content manually, discarding and/or updating outdated pages.

To make this easier I want to add a textblock to the top of each existing page, specifically a template with a notice that this page hasn't been migrated or discarded yet and a category where all these pages are collected (e.g. category:migration_pending). Every user should then look through the pages he is responsible for, copy the content over to the new wiki and change the template to another one marking the page as migrated (category:migration_done) or discarded (category:migration_discarded). This way it should be possible to get a clean, up to date wiki without forgetting anything important.

share|improve this question
    
    
You may also want to follow area51.stackexchange.com/proposals/49276/wikis –  Michael Hampton Jul 4 '13 at 10:03
    
@MichaelHampton: how would you use that? There is nothing there to replace to begin with –  Gerald Schneider Jul 4 '13 at 10:04
    
Oh, I see, it might be possible with a regular expression. I'll look into it. –  Gerald Schneider Jul 4 '13 at 10:06
    
First tries resulted in PHP errors during the replacement. I'll contact the author of the extension. –  Gerald Schneider Jul 4 '13 at 11:37

1 Answer 1

up vote 0 down vote accepted

The Replace_Text extension was unsuccessful, so I ended writing my own script that uses the MediaWiki API.

I started with the loginscript from here and wrote this script:

#!/usr/bin/php
<?php

$settings['wikiroot'] = "https://server/mediawiki";
$settings['user'] =  "username";
$settings['pass'] =  "password";
// $settings['domain'] = 'Windows';
$settings['cookiefile'] = "cookies.tmp";

$prepend = "{{migration_pending}}\n\n";


function httpRequest($url, $post="") {
        global $settings;

        $ch = curl_init();
        //Change the user agent below suitably
        curl_setopt($ch, CURLOPT_USERAGENT, 'MediaWiki Migration Script 0.1');
        curl_setopt($ch, CURLOPT_URL, ($url));
        curl_setopt($ch, CURLOPT_ENCODING, "UTF-8" );
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLOPT_COOKIEFILE, $settings['cookiefile']);
        curl_setopt($ch, CURLOPT_COOKIEJAR, $settings['cookiefile']);
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
        if (!empty($post)) curl_setopt($ch,CURLOPT_POSTFIELDS,$post);
        //UNCOMMENT TO DEBUG TO output.tmp
        //curl_setopt($ch, CURLOPT_VERBOSE, true); // Display communication with server
        //$fp = fopen("output.tmp", "w");
        //curl_setopt($ch, CURLOPT_STDERR, $fp); // Display communication with server

        $xml = curl_exec($ch);

        if (!$xml) {
                throw new Exception("Error getting data from server ($url): " . curl_error($ch));
        }

        //var_dump($xml);

        curl_close($ch);

        return $xml;
}


function login ($user, $pass, $token='') {
        global $settings;

        $url = $settings['wikiroot'] . "/api.php?action=login&format=xml";

        $params = "action=login&lgname=$user&lgpassword=$pass";
        if (!empty($settings['domain'])) {
            $params .= "&lgdomain=" . $settings['domain'];
        }
        if (!empty($token)) {
                $params .= "&lgtoken=$token";
        }

        $data = httpRequest($url, $params);

        if (empty($data)) {
                throw new Exception("No data received from server. Check that API is enabled.");
        }

        $xml = simplexml_load_string($data);

        if (!empty($token)) {
                //Check for successful login
                $expr = "/api/login[@result='Success']";
                $result = $xml->xpath($expr);

                if(!count($result)) {
                        throw new Exception("Login failed");
                }
        } else {
                $expr = "/api/login[@token]";
                $result = $xml->xpath($expr);

                if(!count($result)) {
                        throw new Exception("Login token not found in XML");
                }
        }

        return $result[0]->attributes()->token;
}


try {
        global $settings;
        $token = login($settings['user'], $settings['pass']);
        login($settings['user'], $settings['pass'], $token);

        $star = "*";
        $dash1 = "-1";

        // get edit token
        $result = httpRequest($settings['wikiroot'] . "/api.php?action=query&format=json&prop=info|revisions&intoken=edit&titles=Main%20Page");
        $result = json_decode($result);
        $editToken = $result->query->pages->$dash1->edittoken;

        // only from namespace: apnamespace=100
        $result = httpRequest($settings['wikiroot'] . "/api.php?action=query&list=allpages&format=json&aplimit=5000&apnamespace=100");
        $result = json_decode($result);
        $allpages = $result->query->allpages;

        foreach ($allpages as $page) {
            echo "Fetching '{$page->title}' ({$page->pageid})...\n";
            $revisions = httpRequest(sprintf($settings['wikiroot'] . "/api.php?action=query&prop=revisions&rvlimit=1&format=json&rvprop=content&titles=%s", urlencode($page->title)));
            $revisions = json_decode($revisions);
            if (isset($revisions->error)) {
                echo "ERROR: " . $revisions->error->info . "\n";
                continue;
            }
            $content = $revisions->query->pages->{$page->pageid}->revisions[0]->$star;
            if (preg_match("/\{\{migration_/", $content)) {
                echo "Already marked ... skipping.\n";
                continue;
            }
            echo "Updating...";
            // add text to content and edit page
            $content = $prepend . $content;
            $post = sprintf("title=%s&text=%s&token=%s", urlencode($page->title), urlencode($content), urlencode($editToken));
            $result = httpRequest($settings['wikiroot'] . "/api.php?action=edit&format=json", $post);
            echo "done\n";
        }
        echo ("Finished (".sizeof($allpages)." pages).\n");
} catch (Exception $e) {
        die("FAILED: " . $e->getMessage());
}

?>

What the script basically does is:

  • Log in with an existing account
  • get a token that allows it to do edit operations
  • retrieve a list of all pages within a given namespace
  • for every page:
    • get the content from the latest revision
    • add a predefined text to the content
    • save the page with the new content

Some additional notes:

  • Make sure the user you want to use exists and has write access to all needed namespaces.
  • Add the user to the "Bot" group. This will remove some restrictions, for example a bot is allowed to do bulk operations in groups of 5000 instead of 500 like a normal user. Not sure if it is necessary for this script, but it can't hurt.
  • When an authentication extension like LdapAuthentication is in use the domain parameter has to be set. And it has to be set to the name of the LDAP source how it is configured in LocalSettings.php, not the actual name of the domain.
  • Disable email notifications before you run the script. Otherwise everyone who watches pages in the wiki will get notifications for every changed page he watches. For me this was $wgEnableEmail = false; and $wgEnotifWatchlist = false; in LocalSettings.php.
  • I ran the script from the command line, it isn't fast and takes a couple of minutes for more than 1000 pages. It would definitely run into a timeout if I ran it over the webserver.

Last, but not least, the template migration_pending I added to MediaWiki:

{|class=warningbox
 | [[Image:Emblem-important.png]]
 | This page hasn't been audited yet, the information on it could be outdated. If you are responsible for this page, please check it's content. If it is still current, add it to the new wiki and change this template to <nowiki>{{Migration_done}}</nowiki>. If the information on this page is not needed anymore change the template to <nowiki>{{Migration_discarded}}</nowiki>
 |}

[[Category:MigrationPending]]

This uses a CSS class for tables we had been using before and adds the page to a specific category. I added similar templates as migration_done and migration_discarded with the corresponding categories.

share|improve this answer
    
Downside I noticed yesterday: This breaks #REDIRECT pages, which is expected in the first line. Haven't looked for a fix for this yet. –  Gerald Schneider Aug 1 at 12:44

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.