|the_only_jack 553a1fe01e||2 months ago|
|logos||2 months ago|
|README.md||2 months ago|
|logo.png||2 months ago|
|logo_small.png||2 months ago|
Dumping ED: howto
These are notes on how to dump and import ED. This howto assumes you are running a *NIX os.
Getting the dump
- If you don't have already, install Python 3.
lxmland any other Python modules required. The dump tool should scream at you if you are missing something.
- You need the Mediawiki dump tools. Either grab them from my fork or the original ones. My fork saves the image list so you can resume grabbing image urls. By grab I mean
- In a terminal, go to the cloned directory.
- The first time, use the following command:
python3 dumpgenerator.py --xml --images --api https://encyclopediadramatica.online/api.php
- This should create a new dump folder, named like this:
- Whenever you resume, use the following command:
python3 dumpgenerator.py --xml --images --resume --path=encyclopediadramaticaonline-XXXXXXXX-wikidump/ --api https://encyclopediadramatica.online/api.php
Be sure to replace the X's with the aprropriate date.
- While dumping the xml, or the images, the wiki might throw an error, or you might temporarily lose connection. If you do not want to babysit the dumping script, use the following command:
until ./resume.sh; do echo "crashed with code $?. Respawning in 1800s..." >&2 sleep 1800 echo "finished waiting, respawning now" done
python3 dumpgenerator.py --xml --images --resume --path=encyclopediadramaticaonline-XXXXXXXX-wikidump/ --api https://encyclopediadramatica.online/api.php, obviously.
This process can take whole days to complete. Afterwards, the folder
encyclopediadramaticaonline-XXXXXXXX-wikidump weighs approximately
56 gigabytes (as of October 2023), and contains everything you need to restore the wiki.
Importing the wiki
This is the more technical part. Don't hesitate to get help from OpenAI. In order to run a mediawiki, you will need to install and enable mariadb and apache2 services. You will need to install
php8 and might need
lua5.1 (you can try different php versions, however if you need lua, it must be 5.1).
Installing a mediawiki can differ a lot from system to system and is left as an exercise to the reader, the process is very well documented on the mediawiki wiki.
Tip: every time you modify a config or install a missing dependency, make sure to restart the whole apache service.
Tip: on certain systems, the apache service is called
Steps to take once you have set up a blank wiki instance:
- Add the following after the
$logoPath = "logos";.
- Change the
$wgLogos = [ '1x' => "$wgResourceBasePath/$logoPath/Encyclopedia_Dramatica_logo.svg", 'icon' => "$wgResourceBasePath/$logoPath/Encyclopedia_Dramatica_logo.svg", ];
- Add the following right after:
$wgFooterIcons["poweredby"]["mediawiki"] = [ "src" => "$wgResourceBasePath/$logoPath/powered_by_drama.png", "url" => "$wgServer", "alt" => "Powered by Drama", ]; $wgFavicon = "$wgResourceBasePath/$logoPath/ED_favicon_saber.png";
- Change the
$wgUseImageMagick = true;line to
$wgUseImageMagick = false;. Note, this might not be necessary. If images are not broken, you can leave this at
- Add the following to the end of the file:
wfLoadExtension( 'Scribunto' ); $wgScribuntoDefaultEngine = 'luastandalone'; $wgScribuntoEngineConf['luastandalone']['luaPath'] = '/usr/bin/lua5.1'; wfLoadExtension( 'SyntaxHighlight_GeSHi' ); wfLoadExtension( 'ParserFunctions' ); $wgPFEnableStringFunctions = true; $wgScribuntoUseGeSHi = true; wfLoadExtension( 'YouTube' ); wfLoadExtension( 'TemplateStyles' ); wfLoadExtension( 'MobileFrontend' );
- You may need to edit
$wgScribuntoEngineConfaccordingly to your setup.
- Download and extract the following extensions to their respective folders:
- Note: you will need some of these extensions before running the import scripts, so be sure to follow in order.
- Copy the logos folder (the one with the images) to the root of your mediawiki installation.
Performing the import:
- Open a terminal and go to the mediawiki folder (the one which contains
- Use the following command to import articles and edits (be sure to replace the example path with the actual xml path):
cd maintenance && php run.php importDump --conf ../LocalSettings.php /path/to/dump/encyclopediadramaticaonline-XXXXXX-wikidump/encyclopediadramaticaonline-XXXXXX-history.xml --username-prefix="" --report 100
- Once this is done, you can import the images using the following command:
cd maintenance && php run.php importImages.php /path/to/dump/encyclopediadramaticaonline-XXXXXXXX-wikidump/images --comment-ext=desc
- Note: importing the xml might take a couple days. Importing images can take a day too. You can interrupt and resume the scripts with no problems.
General note: these instructions are not set in stone and will inevitably rot. Use your brain and adapt. Ask people and the bot.