• View
• Edit
• History
• Attach
• Print
• Backlinks

WikiSh

If you are upgrading from a pre-2008-07-31 version please re-read the installation & configuration instructions carefully. There have been some important changes which are required during the installation process. In particular the $WikiShWriteList array (with the corresponding text arrays if you use them) have been replaced with SecLayer capabilities and the control panel has been moved to a separate PHP script (WikiShCL.php) which must be included if you use this capability.

See The WikiSh Examples Page for some mini-applications and working examples of WikiSh scripts.

See The WikiSh Tutorial for a tutorial (starting very basic!) on using WikiSh.

Description

A scripting language emulating linux shell tools to accomplish text & page manipulation and form validation.

WikiSh is not so much a recipe itself as it is an environment for accelerated recipe development. Some one-off commands will be useful in and of themselves in certain circumstances, but generally there is going to be some design/development/testing cycle that goes into most usage of WikiSh -- as such it needs to be seen as a development "platform" in which you can very rapidly accomplish relatively complex operations while not having to worry about authorizations, reading/writing files, and things like that because WikiSh will handle that for you. If you are a relatively inexperienced PmWiki author with limited programming experienced you may well be better off looking at the example page to see if there is an existing solution for you rather than trying to develop it on your own. If, on the other hand, you have programming experience (especially if you already know bash scripting!) then WikiSh has huge potential to accelerate your efforts in developing solutions for your PmWiki site.

Questions answered by this recipe

• Where can I find a scripting language/macro language that I can work with on a page within my wiki-site without going to php & ftp all the time?
• How can I pull in certain lines from another page into the current page (or writing to another page), selecting lines by pattern or line number or etc?
• How can I do search/replace either on those pages (i.e., changing those pages) or on the lines I pull into this page from other pages?
• How can I count lines/words/characters in wiki pages or certain lines within those pages?
• How can I delete a page (or a large number of pages) quickly?
• How can I rename a page or move a bunch of pages into a different group?
• How can I copy one page to another or many pages from one group into another group?
• How can I copy source from many pages into simple textfiles for external processing?
• How can I combine several pages into a single page so that the actual text is in that page (not included via directives)?
• How can I programmatically manipulate wiki pages (and/or simple text files) using in-page (or form-based) mark-up?
• How can I sort a bunch of lines within a page (or pages) by various fields?
• How can I compare 2 different pages to see if there are differences? (Verify a copy, etc)
• How can I validate the input fields from a form without using underlying PHP?

Common misconceptions / concerns

• Can I use WikiSh even though my server is not linux-based or even though I don't have command-line access to my server?
  • YES! WikiSh is a recipe written entirely in PHP with no other pre-requisites other than PMWiki version 2.2 beta or above.
• Does WikiSh open my system up to malicious users, spammers, hackers, etc.
  • If WikiSh is properly configured then there should be no risk to your system. Obviously proper configuration is a significant concern - please note the security precautions below.
  • WikiSh is designed primarily as an ADMINISTRATIVE tool and as such it is recommended that only admins have general write privileges. However, WikiSh is a very helpful tool even without write privileges (validating forms, massaging text, searching, etc.).
  • WikiSh categorically respects and enforces PMWiki auth privileges.
  • Any recipe must be appropriately administered to reduce the risk from malicious users. WikiSh is a general purpose tool with significant capabilities and administrators must take correspondingly greater care to be sure that write privileges are limited appropriately.
  • See security section below for more details.
• WikiSh is a relatively large PHP program (over 100k). Won't that hurt performance on my system?
  • WikiSh (with all PHP programs) is loaded only server-side rather than being downloaded to clients. There will be some "hit" on resources but on a server with adequate capabilities the "hit" should be negligible. (Hmmm... Compile time could be affected, I suppose - haven't noticed it, though.)
    • If someone has further information on this I would be interested -- I'm no performance expert...
  • If you are really concerned about this issue then you could load the script only when you are pages within the WikiSh group so you can still have the capabilities within the ControlPanel and etc. (See installation option 3.D)
• Doesn't WikiSh go against several of the underlying philosophies of PmWiki? Make it easy on authors, don't add unnecessary features, that kind of thing?
  • It depends how the administrators/authors use it. With the addition of functions (as of the 2008-07-31 release) potentially complex wikish scripts can be isolated from end-users simply by defining a function on another page and then calling that function within the current page. So, yes, you could go against the philosophy if you chose to use it in a complex manner within sight of non-technical authors. But you can also easily hide that complexity just as you hide the complexity of the underlying PHP of pmwiki in "invisible" PHP files...

Installation:

A simple, recommended configuration is described here. WikiSh provides a great deal of configuration control and flexibility. For more details please see WikiSh Installation & Configuration.

With this configuration all users will be able to make use of WikiSh and will have read access to all files and limited write access to only to pages in the Test group. Admins will have full WikiSh capabilities. The control panel (command line) will be available only to pages within the WikiSh group. (Again, you can choose to give less privileges or more privileges -- this recommended configuration is somewhat middle-of-the-road. See WikiShConfig for more details on other options.)

1. Download WikiSh.zipΔ
2. Unzip and place WikiSh.php, WikiShCL.php, SecLayer.php, and toolbox.php in your cookbook folder (usually pmwiki/cookbook)
3. Install in config.php as follows:

include_once("$FarmD/cookbook/toolbox.php");
include_once("$FarmD/cookbook/WikiSh.php");
if ($group == 'WikiSh')
   include_once("$FarmD/cookbook/WikiShCL.php");
include_once("$FarmD/cookbook/SecLayer.php");
$EnableWikiShWritePage = true;
$EnableWikiShCreatePage = true;
$EnableWikiShOverwritePage = true;
if (CondAuth($pagename, "admin")) {
   slAddAuth($wshAuthPage, "*.*", "read,create,insert,overwrite,append,prepend,attr,delete");
   $EnableWikiShDeletePage = true;
   $EnableWikiShChmod = true;
} else {
   slAddAuth($wshAuthPage, "*.*", "read,create");
   slAddAuth($wshAuthPage, "Test.*", "insert,overwrite,append,prepend,delete");
}
# cookbook/powertools.php is *very* helpful in the wikish environment but is optional.  
# If you decide to install it you will need to go that page and download it and install
# it as follows:
#include_once("$FarmD/cookbook/powertools.php");

The configuration above, when placed in your config.php, is sufficient to start using WikiSh.

If you want to further utilize the capabilities of SecLayer.php with this recommended configuration, then read on below...

If you wanted to be able to manipulate your page authorizations by editing a page rather than through direct editing of the config.php you would use this code within config.php instead of that above:

include_once("$FarmD/cookbook/toolbox.php");
include_once("$FarmD/cookbook/WikiSh.php");
if ($group == 'WikiSh')
   include_once("$FarmD/cookbook/WikiShCL.php");
include_once("$FarmD/cookbook/SecLayer.php");
$EnableWikiShWritePage = true;
$EnableWikiShCreatePage = true;
$EnableWikiShOverwritePage = true;
slParsePage($pagename, "SiteAdmin.WikiShAuth#aliases", $wshAuthPage);
if (CondAuth($pagename, "admin")) {
   slParsePage($pagename, "SiteAdmin.WikiShAuth#admin", $wshAuthPage);
   $EnableWikiShDeletePage = true;
   $EnableWikiShChmod = true;
} else {
   slParsePage($pagename, "SiteAdmin.WikiShAuth#NONadmin", $wshAuthPage);
}
# cookbook/powertools.php is *very* helpful in the wikish environment but is optional.  
# If you decide to install it you will need to go that page and download it and install
# it as follows:
#include_once("$FarmD/cookbook/powertools.php");

This text should be placed in the page SiteAdmin.WikiShAuth:

[[#aliases]]
edit = append,prepend,insert,overwrite,create
all = edit, delete, read, attr

[[#admin]]
*.*:all

[[#NONadmin]]
*.*:read
Test.*:edit

• SecLayer does not specify what the authorization layers are, but WikiSh honors the following 10 possible authorizations:
  • append -- allow lines to be appended to the end of a page
  • prepend -- allow lines to be prepended before existing lines on a page
  • insert -- allow lines to be inserted between other lines on a page (nothing is deleted)
  • overwrite -- allow a page or a section of a page to be overwritten
  • create -- allow a new page to be created
  • read -- allow a page to be read
  • delete -- allow a page to be deleted
  • attr -- allow the passwords on a page/group to be changed (used only in chmod)
  • forceread -- override PmWiki 'read' authorization (allow read) (use with care and only enable on pages which are edit-protected)
  • forceedit -- override PmWiki 'edit' authorization (allow edit) (use with care and only enable on pages which are edit-protected)
• Aliases such as "edit" and "all" and "none" are entirely at your discretion. You can name your aliases Bob & Larry if you want, but normally a more meaningful name is helpful. :-)

4. Take special note of the security section below. WikiSh is a sharp tool enabling you to get the job done, but you can also "cut your hand" with that same sharp tool if you are not very careful. The warnings in that section must be carefully noted. DO NOT GIVE WRITE PRIVILEGE TO ANYONE YOU DO NOT TRUST! A malicious user given inappropriate write permissions could cause a lot of damage with WikiSh.
5. (optional) Create a page (recommended: WikiSh.ControlPanel) containing the markup (:wikish_controlpanel:) or (:wikish_command_line:). A page like this is not a necessary part of installation, but it is the recommended way to work with WikiSh. Note that you will need cookbook/WikiShCL.php to be included for that page - if you want this markup always available then you can include this line in your config.php:

   include_once("$FarmD/cookbook/WikiShCL.php");

Security:

The default installation of WikiSh (if no changes are made to any $EnableWikiSh___ variables and no calls are made to slAddAuth() nor slParsePage) does not allow any writing to any page. Normally WikiSh will either be installed in this read-only mode or else giving administrators write authorization but not non-admin editors (or perhaps giving non-admin editors write authorization to just a few specific pages or groups). Giving broader write permissions to non-admin personnel opens you up to potential risk as significant damage can be done in a very short time via WikiSh.

Security Levels

Recognizing the potential risk a scripting language presents in a collaborative environment such as a wiki page, WikiSh implements 4 different layers of security, particularly for write authorization to pages. ALL layers must be passed for a given page write. If a single layer is not passed then the write operation will not be allowed.

• LAYER 1: PmWiki has a fairly comprehensive set of authorizations built in and these have been expanded by various other recipes. WikiSh always enforces authorizations set by PmWiki (unless an administrator has specifically enabled a "forceread" or "forceedit" authorization for a given page). If a user wouldn't be able to hit the "edit" button and make a change on a given page then they won't be able to write to that page through WikiSh. (This layer of security is implemented through operating system file permissions for text file access.)
• LAYER 2: ALL writing can be turned on or off with one "switch" which is the $EnableWikiShWritePage variable. If it is set to false then WikiSh will never write to any page. If it is set to true then this layer will be passed. (This layer is implemented for text files both for reading and writing through the $EnableWikiShTextRead and $EnableWikiShTextWrite. The default is for both of these to be false.)
• LAYER 3: Writing can be broken down into 2 types: either changing existing pages or creating new pages. Authorization to do either of these activities can be turned on or off with the variables $EnableWikiShCreatePage and $EnableWikiShOverwritePage. If they are set to true then the respective authorization is given; if set to false then this layer of security will not be passed. (This layer does not exist for text file access.)
• LAYER 4: The administrator can configure a fine degree of control using the capabilities of SecLayer (slParsePage(), slAddAuth(), etc.)

See WikiSh Installation & Configuration for more details on configuration and security.

Quick instructions for existing shell scripters

• Plan on using the control panel for any one-off commands you want to run or test.
  • Really, I'm serious. If you don't use it you will waste a TON of time.
• When you are going to embed commands in pages, take note of these sections in particular:
  • wikish, and particularly the {(wikish ... source PAGENAME#SECTION ...)}
  • wikish_button for making commands executable via a button on the page
• Do not assume that the command or option that you need is available. Only a small subset of shell functionality has been implemented. If you find a very useful feature that's not implemented, drop me a line or include a note in the talk page and I'll see what I can do about implementing it.

General Usage:

• COMMAND
  • See specific commands below for more information (a more detailed section follows in detailed commands.
• OPTIONS
  • Options can be set anywhere in an MX command via option=value. If you wish to use the particular WikiSh syntax (recommended) then most option settings begin with a hyphen or a double-hyphen. More details can be found in the options section.
• PAGESPEC
  • From 0 to n pages can be specified. Wildcards (* and ?) are honored. Separate by spaces. If no group is specified the current group is assumed. Precede a pagespec with TEXTFILE-- to refer to a text-file instead of a wiki-page.
  • More details can be found in the Files section below.
• OUTPUTSPEC
  • Normally the output produced by any MX is placed in exactly the position on the page where the MX command appeared. The output replaces the MX command.
  • However, there are times when you want the output of the command to go somewhere else, particularly into another page. In this case you append an OUTPUTSPEC to the very end of your command.
    • In wikish "lingo" (i.e., the way shell programmers refer to it) this process of sending output somewhere else is referred to as "redirecting the output" or "redirecting standard out" or abbreviated "redirecting stdout". Note that the word "redirect" in this context has nothing to do with your browser opening another page -- it simply means the the output of the command is placed (directed to) somewhere else. Because of this you will later on see options named "stdout" -- this refers to "standard output".
  • OUTPUTSPEC normally takes the form of a right-angle-bracket (">") followed by a pagename (this will replace the entire content of that page with the output of the given command), but this can be modified in various ways. More details can be found in the OUTPUTSPEC section below.

These are the COMMANDS that are currently available:

See in particular {(wikish ...)} as it can (should?) be used as the main interface to the other commands...

• basename (from a full path, calculate the last filename portion)
• cat (conCATenate page(s) -- simply list their contents)
• chmod (CHange MODe -- change attributes of a page or group)
• cp (CoPy page(s) to another page or to another group)
• cut (cut specified fields or character ranges)
• dirname (from a full path, calculate the directory portion)
• diff (compare 2 pages)
• echo (simply produce MX output via arguments)
• fetchmail (get mail from POP3)
• grep (search for a regular expression in a page(s))
• head (list first n lines on page(s))
• ls (LiSt files in various formats)
• mv (MoVe page(s) to another page or to another group)
• null (do nothing - /dev/null)
• read (read values from a page one-line-at-a-time for looping purposes)
• rm (remove - will be done by replacing text with "delete" and calling UpdatePage()
  • Note the need to set $EnableWikiShRemove and $EnableWikiShRemoveFully to get this capability.
• sed (extract sections of a page(s) by pattern or by line number, do search/replace on a page(s))
• mailx (send email)
  • Note the dependency on WikiMail for this MX
• set (set variables which will be honored by any WikiSh MX)
• sort sort lines lexicographically (options to divide into keys, ignore whitespace, dictionary order, etc.)
• tail (list last n lines on page)
• test allow various boolean tests (primarily for use with if and while control structures in wikish)
• uniq (print only unique lines, only duplicate lines, count of duplicates, etc.)
• wc (count lines, words, or character in a page(s))
• wikish (generic expression processor - can implement any MXes and allows limited flow control via the following:
  • if BOOL-EXPR; then; EXPR-LIST; else; EXPR-LIST; fi; (nested to your heart's content)
  • while BOOL-EXPR; do; EXPR-LIST; done;
  • for VAR in LIST; do; EXPR-LIST; done;
  • source PAGENAME [args];
• (xargs implemented by NOT including a hyphen prior to another MX - will be read as another list of files -- see section "Files and Nested Markup Expressions (MX)" below)
• future: cd (ChDir -- not sure about this from a security standpoint...)
• future: prettypage (this is not exactly SH [although it has a near relative] but it would be nice to apply fmt=x from pagelist to a result pagelist...)
• future: allow test to be invoked via [...]

Meta-Commands (non-shell)

• wikish_button (display a button on a page which, when pressed, will execute a WikiSh MX)
• wikish_controlpanel (This is (:...:) markup rather than MX - provides a form-based way to run WikiSh commands, keep track of history, save to favorites, etc.)
• wikish_active (able to activate/deactivate most wikish commands)
• once (an MX which will return a status of 0 one time, 1 all subsequent invocations. Use in conjunction with wikish_active to make sure a command on a page only gets run once)
• wikish_form (create a quick form, help with redirects, set fields to vars)

Alternate invocations

You can also invoke an MX using {earlymx(command ...)} if you need to interact with PVs before they are interpolated on that page. This would normally be by use of the {earlymx(set --pv -s VAR = (command ...))} to set {$VAR} to something.

Specifying options

Options are of 2 types: boolean and those accepting a value. You can also specify in different ways depending on whether you are using short- or long- options. (Short are just single-character flags while long may be an entire word.

Examples of option-setting:

• (sed -n ...) -- sets a boolean 'n' flag to true
• (grep --ignorecase ...) -- sets a boolean 'ignorecase' flag to true
• (cut -d: ...) -- sets the 'd' option to a colon
• (grep --line-prefix:"FILENAME: " ...) -- sets the 'line-prefix' option to 'FILENAME: ' (note the use of a colon rather than equal-sign -- this is necessary because of the way MXes naturally process other options - see below)

You can also specify options simply as option=value.

Generic Options (these work on nearly all commands, at least where they make sense)

As with other options you can either use the preferred WikiSh option setting syntax (as specified below) or you can use the PmWiki syntax of OPTION=VALUE (no prefixed double-hyphen). So you can either run {(echo --timeout:60 "hello, world")} or you can run {(echo timeout=60 "hello, world")} to do the same thing.

• --timeout:n -- it resets the 30-second timelimit on the host, allowing more or less time before PHP times out; you may get more or less mileage than I have... If you are doing any significant read/write operations and your host isn't too quick then you can save yourself headache by setting this to a higher number. Note also session capabilities within the {(read ...)} MX and the variable ${SECONDSLEFT} for other ways to work around this timeout. Technical note: setting this option calls the PHP function set_time_limit($opt['timeout']);.
• --file_prefix:"value-to-be-substituted" (see note below) (This value, after substitution, will occur on the line prior to each file/page processed.)
• --line_prefix:"value-to-be-substituted" (see note below) (This value, after substitution, will occur on each line of output, preceding the output on that line.)
• --line_suffix:"value-to-be-substituted" (see note below) (This value, after substitution, will occur on each line of output, following the output on that line.)
• --stdout:pagename (Beware as you can easily overwrite a page!!!) (deprecated synonym for the preferred OUTPUTSPEC)
• --stdout_type:wiki|text (to write out to a wiki page or a text file -- default is wiki page if not specified (note requirement to set $EnableWikiShTextRead and $EnableWikiShTextWrite if you want to use simple text files - note security risks above. There is no reason to use this option unless you are trying to send output to textfiles.)
• --stdout_loc:X (Deprecated. use OUTPUTSPEC to set this. If used it should contain a pattern which will be found on the output page and then output will be replaced either before or after that line, depending on the setting of stdout_op below.)
• --stdout_op:X (Deprecated. use OUTPUTSPEC to set this. If used it should be either < or > to indicate before or after the stdout_loc value.)
• --tee (Normally if stdout is set (either explicitly or by use of OUTPUTSPEC) then all output goes to the specified page/file and no output will be returned from your MX. If --tee is specified then this will make your standard output go to BOTH the stdout file as well as being returned from your MX. Without this option the stdout will "swallow" all output and no output will be returned from your MX.
• newline=x This allows you to change the normal newline-separated-line behavior to being separated by something else. (For instance, you could use this within (grep -l ...) to make it a CSV list of files as an input to a {(pagelist ...)})
• markup=code (Actually currently it can be set to anything and it will result in a simple substitution of all curly-braces and parenthesis with their octal codes to prevent any further markup being processed. If there's other markup "leaking" through let me know and I'll expand the list of substitutions. FUTURE: This can be expanded to other ways of handling markup.
• --debug:n (sets the debug level. Everything this level and above will be printed, so a lower number is more detailed. 5 turns nearly all debugging off and is the default as long as I remember to set it before I release... ;) 1 turns on all debugging and will significantly slow down processing. )
• --stderr:LOC (Loc can be either "messages" or "echo" or "/dev/null" ["null" or "nul" synonyms for the latter] [default "echo"]. Error messages will either be outputted into the (:messages:) section ("messages") or echoed at the top of the screen ("echo") or completely suppressed ("/dev/null" and siblings). Note that if you suppress the error messages ... how do I say this ... you won't see them and they may be important!
• --xargs (says that any piped input to this command should be treated as a list of files to be processed rather than an inline file)
• --list:X -- This will set an option of which pages should be "stripped out", using the same semantics as the "line=X" option to (:pagelist ...:). Valid values are "normal" and "all" unless your system has explicitly set others. This option, given for a particular command, will override any ${LIST} set elsewhere.
• --refpage:PAGE -- This will use PAGE as the reference page for PVs which are substituted. (Same as setting ${REFPAGE} but quicker and easier.)
• --display OR --html -- Either of these options will cause special characters to be displayed in a more standard way on output. But you should not use this on output that will be put into source on a page -- it should be used only for output that will be immediately displayed on a page. (This is particularly important for markup such as [[<<]] which will only work if the characters are in display mode.) (Technical note: use of this option causes PVSE() function to be called on the return value of the MX.)
• --encrypt -- only available if WikiShCrypt is installed - see section on encryption/decryption
• --decrypt -- only available if WikiShCrypt is installed - see section on encryption/decryption
• --passwd -- only available if WikiShCrypt is installed - see section on encryption/decryption

• Note that in the above options the "value-to-be-substituted" is a simple string, but certain strings will automatically be replaced:
  • PAGENAME (use --file_prefix:"[[PAGENAME]]" (or some other prefix/suffix) if you want a link)
  • PAGETITLE
  • LINENO (only on line-prefix)
  • For instance, you could use the --line-prefix option to put the pagename and line number before each line of output from grep:

{(grep --line_prefix:"PAGENAME: LINENO: " "mysearchstring" MyGroup.*)}

• Or you could create an entire table in a single command:

{(grep --file_prefix:"|| border=1 width=90%" --line_prefix:"|| LINENO||" --line_suffix:"||" "mysearchstring" MyGroup.MyPage)}

Encrypting and Decrypting Text on Read/Write of Pages & Files

See WikiShCrypt for installation instructions, more details, instructions on how to encrypt, etc.

Specifying Pages & Files and using nested MarkupExpressions (MX)

Files can be specified with full wildcards. If you do not specify a group (i.e., have a "." in the filename specification somewhere) then the current group will be assumed. (i.e., if you are in the page group.mypage and you ask for * you will get group.*)

You can specify the output of one MX to be the input of another by specifying a single hyphen prior to the inner MX (that tells the outer program that the next argument is a nested MX instead of a filename. For instance, if you wanted the to count the lines between #STARTSECTION and #ENDSECTION you might do the following:

{(wc -l - (sed -n '/#STARTSECTION/,/#ENDSECTION/p' filename))}

That usage (above) means that the output of one MX will be used as if it were the CONTENTS of a file. If you are interested, instead, in having each line of output from an inner MX become a separate argument in the outer MX then simply do NOT prepend the - before the inner MX. For example:

{(wc -l (grep -l "hello" main.*))}

That will find all files which match the regular expression "hello" and then give you a count of the number of lines in each of those files. This should be clearly differentiated from this:

{(wc -l - (grep -l "hello" main.*))}

which will count the number of files which contain a match of the pattern "hello".

Note that backquotes accomplish much the same thing but guarantee the expected order of execution. Putting it in a parenthesized "sub-MX" means it will get executed before any other commands in your MX. Putting it in backquotes means it will get executed after all other preceding commands but before the command it is a part of (in other words, backquotes do what you expect while parentheses will usually mess you up).

A section within a page (only for reading) can be specified with the expected syntax PAGENAME#SECTION#ENDSECTION and it will only read that section. This works for reading, but not for writing. (The other section syntaxes that (:include ...:) supports are also supported here - I'm reusing the same code.)

Virtual pages for increased speed on temporary files

A much quicker and somewhat persistent virtual file system is available through in-memory files stored in your session. Any page created in the "Session" or "Temp" or "Virtual" groups will automatically be used as an in-memory file (this is configurable through setting the $WikiShSessionPagePat array). You can also specify SESSION--Pagename to force a given page to be an in-memory page. Note that these virtual pages are available only within WikiSh -- you cannot try to browse them with PmWiki. They are significantly faster in reading and writing than disk-based files. Thus they are ideal for an application which will write to a page many times (appending, for instance) -- do 100 writes to an in-memory file and then at the end of your script copy that page into a normal PmWiki page once again. These virtual pages will exist as long as your session is valid. (Using a virtual page instead of writing to disk can save from 300% to 1700% on scripts which are appending single lines to a file many times.) Just don't forget to copy it from virtual space to a "real" page before you are done!

OUTPUTSPEC -- sending the output of an MX command to another page

• Commands which produce output can have that output redirected to another page via OUTPUTSPEC. The following functionality is supported:
  • >pagename - write the output to that page, overwriting/replacing the existing contents of the page entirely.
    • EXAMPLE: You have a page named Foo.Bar and it contains the words "I think therefore I am". You run the MX command {(echo "Hello, World" >Foo.Bar)}. After it runs the text "I think therefore I am" has been completely replaced and the page Foo.Bar contains only the text "Hello, World".
  • >>pagename - write the output to the page, appending the new output to the end of the existing content of that page. (This is a shortcut for >pagename>_END as documented in the following outputspec.)
    • EXAMPLE: As above but use this MX instead: {(echo "Hello, World" >>Foo.Bar)}. (Note TWO right-angle-brackets instead of 1.) After the MX is executed the contents of Foo.Bar will be "I think therefore I am" on the first line and "Hello, World" on the 2nd and last line. You have appended the text to the end of the page.
  • >pagename>pattern - write the output to the page immediately AFTER the first line matching that (regex) pattern (_END or _BOTTOM are magic patterns matching the last line)
    • NOMENCLATURE: Think of the second right angle bracket as an arrow pointing to the right or AFTER the pattern
    • EXAMPLE: You have the page Foo.Bar which contains 3 lines. Line 1 is "abc", line 2 is "def" and line 3 is "ghi". You execute the MX {(echo "Hello, World" >Foo.Bar>def)}. After the MX executes the line "Hello, World" will be inserted immediately after the line matching the pattern "def". Thus line 1 will be "abc", line 2 will be "def", line 3 will be "Hello, World", and line 4 will be "ghi".
    • EXAMPLE: To repeat the example from the >>PAGE outputspec above: you start with "I think therefore I am" in Foo.Bar. You execute the MX {(echo "Hello, World" >Foo.Bar>_END)}. After the MX is executed the contents of Foo.Bar will be "I think therefore I am" on the first line and "Hello, World" on the 2nd and last line. You have appended the text to the end of the page.
  • >pagename<pattern - write the output to the page immediately BEFORE the first line matching that (regex) pattern (_BEGIN or _TOP are magic patterns matching the first line)
    • NOMENCLATURE: Think of the left angle bracket as an arrow pointing to the left or BEFORE the pattern
    • EXAMPLE: You have the page Foo.Bar which contains 3 lines. Line 1 is "abc", line 2 is "def" and line 3 is "ghi". You execute the MX {(echo "Hello, World" >Foo.Bar<def)}. After the MX executes the line "Hello, World" will be inserted immediately before the line matching the pattern "def". Thus line 1 will be "abc", line 2 will be "Hello, World", line 3 will be "def", and line 4 will be "ghi".
    • EXAMPLE: You have the page Foo.Bar containing the text "I think therefore I am". You execute the MX {(echo "Hello, World" >Foo.Bar<_TOP)}. Your text "Hello, World" has been prepended to that page so line 1 is now "Hello, World" and line 2 is "I think therefore I am".
  • >pagename#section - write the output to the page replacing all text in the given section
  • >pagename$:varname - write the output to the ptv named "varname" on the page specified. Format will be standard text - varname:value. (Note that Toolbox must be installed and available for this capability to be present)
  • >pagename$varname - write the output to the ptv named "varname" on the page specified. Format will be hidden - . (Note that Toolbox must be installed and available for this capability to be present)
  • >pagename$:(varname) - write the output to the ptv named "varname" on the page specified. Format will be hidden - . This is an alternative syntax to above, perhaps more intuitive, but it requires quoting in order to protect the parentheses from the MX engine; for that reason this syntax is deprecated. (Note that Toolbox must be installed and available for this capability to be present)
  • >pagename$::varname - write the output to the ptv named "varname" on the page specified. Format will be definition list - :varname:value. (Note that Toolbox must be installed and available for this capability to be present)

• The description of OUTPUTSPEC given in the above outline is closely related to how the linux shell redirects output. However, that's not the way pmwiki administrators and editors are used to specifying options in markup expressions or pagelists or whatever. Therefore there is an alternate (although somewhat deprecated by the shell purists <wink>) way of specifying where the output of an MX should be placed:
  • stdout=PAGE

    • EXAMPLE: From the first example above you could use this MX: {(echo stdout=Foo.Bar "Hello, World")}. Since stdout_op and stdout_loc are not set then the output of this command will replace any existing content on the page Foo.Bar.

  • stdout_op=OP where OP can be < or > (if stdout_op is set then stdout_loc is required) indicating before or after the line/location specified via stdout_loc
  • stdout_loc=LOCSPEC where LOCSPEC is a regex pattern to be matched in that file or one of the "magic" patterns _TOP, _BEGIN, _BOTTOM, _END. This identifies a line in relation to which the output will be placed either before or after depending on the value specified in stdout_op.
    • The following examples are all exactly above in the >page, >>page, etc, but now modified to use the pmwiki-friendly syntax...
      • EXAMPLE: As above but use this MX instead: {(echo stdout=Foo.Bar stdout_op=> stdout_loc=_END "Hello, World")}. After the MX is executed the contents of Foo.Bar will be "I think therefore I am" on the first line and "Hello, World" on the 2nd and last line. You have appended the text to the end of the page. (You could have used _BOTTOM instead of _END -- they are synonyms because otherwise nobody can ever remember which one was the right one.)
      • EXAMPLE: You have the page Foo.Bar which contains 3 lines. Line 1 is "abc", line 2 is "def" and line 3 is "ghi". You execute the MX {(echo "Hello, World" stdout=Foo.Bar stdout_op=> stdout_loc=def)}. After the MX is executed the line "Hello, World" will be inserted immediately after the line matching the pattern "def". Thus line 1 will be "abc", line 2 will be "def", line 3 will be "Hello, World", and line 4 will be "ghi". (Notice from the order of arguments in this example and the previous that the specification of options in this 2nd manner can be done anywhere within the MX and it is no longer required to be the last item on the command line.)
      • EXAMPLE: You have the page Foo.Bar which contains 3 lines. Line 1 is "abc", line 2 is "def" and line 3 is "ghi". You execute the MX {(echo "Hello, World" stdout=Foo.Bar stdout_op=> stdout_loc=def)}. After the MX executes the line "Hello, World" will be inserted immediately after the line matching the pattern "def". Thus line 1 will be "abc", line 2 will be "def", line 3 will be "Hello, World", and line 4 will be "ghi".
      • EXAMPLE: You have the page Foo.Bar which contains 3 lines. Line 1 is "abc", line 2 is "def" and line 3 is "ghi". You execute the MX {(echo "Hello, World" stdout=Foo.Bar stdout_op=< stdout_loc=def)}. After the MX executes the line "Hello, World" will be inserted immediately before the line matching the pattern "def". Thus line 1 will be "abc", line 2 will be "Hello, World", line 3 will be "def", and line 4 will be "ghi".
      • EXAMPLE: You have the page Foo.Bar containing the text "I think therefore I am". You execute the MX {(echo "Hello, World" stdout=Foo.Bar stdout_op=< stdout_loc=_TOP)}. Your text "Hello, World" has been prepended to that page so line 1 is now "Hello, World" and line 2 is "I think therefore I am".
• It should be fairly obvious to even a casual observer the way these 2 methods of specifying output redirection are related. If you put a right-angle-bracket followed by the value of stdout followed by the value of stdout_op followed by the value of stdout_loc then you have the preferred wikish syntax given above. It will be replaced internally by the stdout, stdout_op and stdout_loc, but it sure saves you a lot of typing!!!
• You should be aware that all testing of WikiSh by the author is done using the first method of specifying redirection. (I'm just too lazy to do all that typing.) Thus it's possible that an error might creep in and be undetected until you generously test the 2nd method and report the error...

Variables

A relatively flexible variable scheme is implemented through the use of the (set ...) command and surrounding variables to be interpolated with ${...}. The idea (this is just like shell although it may seem a little counter-intuitive if you are more used to PHP or something) is that when you want to SET the value of a variable you do NOT use the ${...} (just the "bare" var), but when you want to USE (i.e., interpolate) a variable then you use the surrounding ${...}. So if the variable name is var then you would set it by saying:
{(set var = 2)}
but you would USE it by saying:
{(echo "the value is" ${var})}

Continuing on...

For example:
{(set i = 8)}
{(echo "my son is " ${i} " years old")}
Continuing on:
{(set i --)}
{(echo "my son who is 1 year younger is" ${i} "years old")}
There is no way an MX function can differentiate between strings surrounded by single-quotes as opposed to double-quotes (or a bare word, for that matter) and so variables are interpolated for everything whenever that pattern is found. The only way to suppress it is to prepend a backslash before the dollar sign:
{(set -v i = 8)} prints "8" as the return value
{(echo "4+4=" ${i})} prints "4+4= 8" as the variable is interpolated
{(echo "dollar sign followed by the letter i: \${i}")} prints "dollar sign followed by the letter i in curlies: ${i}" as the variable is NOT interpolated because of the prepended backslash before the dollar sign.
{(echo "dollar sign followed by the letter i: $i")} prints "dollar sign followed by the letter i: $i" as the variable is NOT interpolated because there are no curlies surrounding the variable name.
Strings are a bit more awkward because of the quoting and the need to limit what characters can be passed to the underlying eval statement. Thus the -s is necessary when using set for strings and no other operations are permitted in that case.

{(set -s sons = Christopher Joshua Jonathan)} {(echo "My 3 sons are named" ${sons})}

While arrays are not formally supported, you can concatenate strings within variable names resulting in a workable implementation of arrays:

for i in 1 2 3 4 5
do
   set a${i} = hello   # Set it
   echo ${a${i}}       # Use it
done

Since "$" and "{" and "}" are not valid characters in a variable name you are guaranteed to get the inner variable replaced first and then the outer variable appears as a valid variable and will be interpolated in turn.

Alternate variable types

• Session variables can be set using the --session option of {(set ...)} and retrieved using a ~ prefix to the variable name like ${~varname}
  • Session variables remain set (sticky) from one load of a page to another rather than starting from scratch each time. But sessions are lost in certain circumstances and so you don't use this as "safe" storage -- basically it's a convenience...
• Request variables (GET and POST and COOKIE) can be retrieved using the ! prefix to the variable name like ${!varname}

Credit to the author httpvariables for the markup idea. Imitation is the sincerest form of flattery. Those of you accustomed to httpvariables will appreciate the parallels.

Variables with existing content

These variables can be used without being set and will provide the following functionality:

Variables which affect other behavior (configuration variables) (ordered by likelihood of needing to set them):

These variables can be set like any other variable, but they will have impact on the way WikiSh runs:

Variable modifiers

There are several ways to modify the value of variables (or in some cases the variables themselves) while interpolating them:

• ${#var} will be replaced not with the value of the variable, but with the length of the value of the variable.
• ${var/srch/repl} will be replaced by the value of the variable after a simple search/replace has been carried out on that value
  • Note that the srch pattern is using globs (wildcard patterns like *.*) rather than fixed strings or regular expressions
• ${var:offset:length} will be replaced by a substring of the value of the variable. You have an offset (0-based) into the string and the length. (Actually I just call substr() directly with whatever is passed in for x and y, so you have the usual capabilities there including negative numbers, etc.)
• ${var:-dflt} will be replaced by the value of the variable unless it is unset or blank in which case it will be replaced by the text following the :-
• ${var:=dflt} will be replaced by the value of the variable unless it is unset or blank in which case it will be replaced by the text following the := AND THAT DEFAULT VALUE WILL BE ASSIGNED TO THE VARIABLE.
• ${var:+alt} will be replaced by the alternate text if the variable is set and non-blank. If the variable is unset or blank then the replacement will be for an empty string.
• ${var#pat} delete text matching the pattern off the beginning of the value (smallest possible match) (glob/wildcard patterns)
  • if ${a} is abc.def.ghi ${a#*.} will be def.ghi (abc. deleted)
• ${var##pat} delete text matching the pattern off the beginning of the value (longest possible match) (glob/wildcard patterns)
  • if ${a} is abc.def.ghi ${a##*.} will be ghi (abc.def. deleted)
• ${var%pat} delete text matching the pattern off the end of the value (smallest possible match) (glob/wildcard patterns)
  • if ${a} is abc.def.ghi ${a%.*} will be abc.def (.ghi deleted)
• ${var%%pat} delete text matching the pattern off the end of the value (longest possible match) (glob/wildcard patterns)
  • if ${a} is abc.def.ghi ${a%%.*} will be abc (.def.ghi deleted)
• ${!prefix*} will return a list of all variable names which begin with that prefix
  • if you have 3 variables ${fooa}, ${fooabc} and ${fooxyz} and you use ${!foo*} then you will get "fooa fooabc fooxyz". If you put these in a variable (via a for loop or similar construct) named i, for instance, then you could get to the value of the variable like this ${${i}}
  • Example follows where the above 3 variables exist and you want to print their values

  for i in ${!foo*}
  do
     echo "The variable ${i} contains the value ${${i}}"
  done
  

Return Status

Each command returns text which goes in place of the MX itself, but in addition to that type of return value there is also an "ERROR STATUS" or "STATUS" which is set and returned by each command. Basically if the value is 0 then everything worked fine. If the value is non-zero then something unexpected happened, or we didn't find what we were looking for, or something like that.

The commands (true) and (false) do nothing except to set this return status to 0 and 1, respectively.

You can access the value of this return status using the variable ${STATUS}.

You can also use it in commands which are testing for a boolean value. For instance, in the main WikiSh command you have commands like (if ... then ... else ...) and the initial condition is determined based on the ${STATUS} value.

{(wikish if true; then; echo YES; else; echo NO; fi)}

Note that ${STATUS} is set by each successive command, so the current value is based on whatever the LAST command was that completed.

Functions

Various commands within WikiSh (or even other non-WikiSh MXes) can be combined within a single "function" which then becomes its own MX.

This capability is significant in order to remain within the "author-friendly" philosophy of PmWiki. Relatively complicated WikiSh scripts can be placed in seldom accessed pages and simple MX functions can be executed in pages where non-technical authors roam.

There are 2 ways to define a function:

• Within a single-line MX (technically it's not an MX, but if it looks like an MX and it smells like an MX...)

{(function abc echo "Hello, World"; if true; then; echo "Logic still works"; else; echo "Uh, oh, we're really in trouble now."; fi)}

• In a piece of source'd script

{(wikish source WikiSh.Functions#abc)}

• If you use the second option (preferred) then you would need to have this section in WikiSh.Functions:

[[#abc]]
function abc()
{
   echo "Hello, World!"
   if true
   then
      echo "Whew! Logic still works"
   else
      echo "Uh, oh, we're really in trouble now."
   fi
}

In both cases, after the function has been defined you can run it just as you would run any other MX:

• {(abc)}
• {(wikish abc)}
• [in the control panel:] abc

It is possible to pass arguments to functions using positional parameters. Your function will see them as ${1}, ${2}, etc. You can see ALL positional parameters using ${*} or ${@}.

[[#greet]]
function greet()
{
   echo "Hello, ${1}!"
   if test ${2} == 'guy'
   then
      echo "Looks like it's boys' night out.  BRAAAP!"
   else
      echo "Ladies present! Guys, cut the belching!"
   fi
}

Then when you call it you can pass arguments (in this case a name and then a gender):

{(wikish source WikiSh.Functions#greet; greet Sam guy; greet Janet gal)}

Hopefully the result would be this:

Hello, Sam!
Looks like it's boys' night out.  BRAAAP!
Hello, Janet!
Ladies present! Guys, cut the belching!

The return status of a function is the return status of the last command which is run within that function.

Note that any functions defined in the page WikiSh.Profile or WikiSh.WikiShRC will be automatically defined whenever WikiSh is active as long as at least one wikish command has been run beforehand (this is most easily accomplished simply by running the function within a {(wikish ...)} context).

Thus if I put this in WikiSh.WikiShRC:

function abc
{
   echo "I know my alphabet. Won't you sing with me?"
}

Then on any page within my site I can run this function by placing this source code on the page:

{(wikish abc)}

Or, if I have already run another wikish MX then I can access it directly:

{(echo foo)}
{(abc)}

In addition to WikiSh.WikiShRC and WikiSh.Profile you can also place group-specific source in WikiSh.GROUP-GroupRC and page-specific function definitions (or variables being set or whatever) in WikiSh.GROUP-PAGE where GROUP and PAGE are the names of the group and page of the page which needs the function definition, etc. All of these pages "understand" at least 3 pmwiki rules: if conditions, include directives, and comment directives. This allows you to include certain pages (or sections) and etc.

• If you wish to configure these startup pages you can set 2 variables:
  • $WikiShRCPages (an array containing the list of pages, perhaps defined using PVs)
    • DEFAULT: array('WikiSh.WikiShRC', 'WikiSh.Profile', 'WikiSh.Cookbook-GroupRC', 'WikiSh.Cookbook-WikiSh')
  • $WikiShRCRules (an array containing the list of rules which will be executed over these pages before they are executed by WikiSh)
    • DEFAULT: array('include', 'comment', 'if')

These startup pages are not limited to function definitions. You could set a variable (${LIST}, for example) or create a temporary page or whatever. It's just that function definitions are probably the most common use of these startup pages. You can place any complex WikiSh code in the startup page (think of it as a per-page configuration) and then in your page all you see is {(wikish function_name argument)}.

Detailed Listing of Each Command

basename (calculate just the pagename/filename portion of a fully qualified pathname/group.pagename)

• {(basename FILE)}
• Options
  • --raw - pass the argument directly to PHP dirname() function (i.e., working with directories and filenames rather than groups and pages)
  • Note that if TEXTFILE-- is prefixed to the pathname then --raw is automatically assumed
• Examples:
  • {(basename Mygroup.Mypage)}
    • returns "Mypage"
  • {(basename TEXTFILE--path/to/file)}
    • returns "file"

cat (conCATenate pages)

• {(cat OPTIONS FILES)}
  • Options are only the generic ones
  • This simply returns the files or nested MXes which are its arguments. It's nice for pulling together results of several commands.
  • EXAMPLE: (cat - (echo || border=1) - (sed --line-prefix:"||PAGENAME||" -n '1,10p' group.file?))
    • It's not quite that simple to create a table, but almost... We're still missing the ending || -- if you want that you have to make it a little more complicated as below:
    • EXAMPLE: (cat - (echo || border=1) - (sed --line-prefix:"||PAGENAME||" -n -e '1,10p' -e 's/$/||/' group.file?))

chmod (CHange MODe - change passwords on groups/pages)

• {(chmod OPTIONS FILES)}
  • Options
    • -p - this will apply to PAGES (this is the default if neither -p nor -g are specified)
    • -g - this will apply passwords to GROUPS
    • --read:PASS - set the "read" password to PASS
    • --edit:PASS - set the "edit" password to PASS
    • --attr:PASS - set the "attr" password to PASS
    • --upload:PASS - set the "attr" password to PASS
    • -a - the passwords will be *added* to existing passwords for this group/page rather than replacing existing passwords. (This behavior is unique to chmod and cannot be done using the usual form-based interface.)
  • This MX sets a password for the specified pages or groups. If neither -p nor -g are specified then -p is assumed. If -g is specified then the assumed -p is no longer assumed. If you want to change both page and group passwords (why?!) then you would need to specify both -p and -g.
  • Note that as in all WikiSh MXes you are responsible to obtain the necessary pmwiki authorizations prior to running the command. In this case you must have attr authorization on any pages or group.GroupAttributes pages that you are attempting to change.
  • EXAMPLE: {(chmod --edit:mysecret Main.HomePage Test.OtherPage GroupA.*)}
• Note that if you use authorizations besides read, edit, attr, and upload you can change the configuration variable $wshPmAuthList to something besides array('read', 'edit', 'attr', 'upload'). You could also use this to limit which passwords people were allowed to change via chmod.

cp (CoPy pages)

• {(cp OPTIONS PAGES ... DESTINATION)}
  • Options
    • --trial - don't actually copy - just tell me what you would have done
    • -q - be quiet about it. Do the copy, but don't tell me what you've done.
  • PAGES - as usual with wildcards or specifying multiple files
  • DESTINATION - if multiple source pages were specified you must specify either a group name (copy those pages to the other group) or a directory for text files (TEXTFILE--dirname)
  • EXAMPLE:
    • (cp groupa.page1 groupb.page2)
      • This will copy groupa.page1 to groupb.page2
    • (cp groupa.* groupb)
      • This will copy all pages in groupa into groupb
    • (cp groupa.* TEXTFILE--mydir)
      • This will place text files containing the source in a directory relative to the main pmwiki directory called mydir

cut (cut characters or fields from each line of input)

• {(cut OPTIONS PAGES ...)}
  • Options
    • -dx - specify a delimiter of x for splitting into fields (default TAB)
    • -fLIST - specify a list of fields to print (fields counted from 1)
    • -cLIST - specify a list of characters to print (characters counted from 1)
    • --ofs:z - specify an output field separator (if not specified it defaults to x from -dx)
  • LIST
    • x - just that character or field
    • x-y - characters or fields from x to y
    • x- - characters or fields from x to the end
    • x,y - characters or fields x and y
    • x-y,z - characters or fields from x to y and z
    • etc.
  • PAGES - as usual with wildcards or specifying multiple files
  • EXAMPLE:
    • (wikish echo "abc\tdef\tghi\na\tb\tc" | cut -f2)
      • This will print def and b (field 2 of each line)
    • (wikish echo "abc def ghi\na b c" | cut -d' ' -f2)
      • This will print def and b (field 2 of each line, fields now separated by a space)
    • (wikish echo "abc def ghi\na b c" | cut -d' ' -f1-2)
      • This will print "abc def" and "a b" (fields 1 & 2 of each line)
    • (wikish echo "abc def ghi\na b c" | cut -c1-2,5)
      • This will print "abd" and "a c" (characters 1 to 2 and 5 of each line)

diff (compare files)

• (diff OPTIONS PAGE1 PAGE2)
  • Options
    • -q - don't show changes, just return ${STATUS} for use in while loops or if conditionals (true if files are identical)
  • EXAMPLE:
    • (diff groupa.page1 groupb.page2)
      • This will compare groupa.page1 to groupb.page2 and print off a diff output
    • (wikish if diff -q groupa.page1 groupb.page1; then; echo "They are the same"; else; echo "They are different"; fi)
      • This will print a quick messages indicating whether the 2 pages are the same or different

dirname (calculate just the directory/group portion of a fully qualified pathname/group.pagename)

• {(dirname OPTIONS FILE)}
• Options
  • --raw - pass the argument directly to PHP dirname() function (i.e., working with directories and filenames rather than groups and pages)
  • Note that if TEXTFILE-- is prefixed to the pathname then --raw is automatically assumed
• Examples:
  • {(dirname Mygroup.Mypage)}
    • returns "Mygroup"
  • {(dirname TEXTFILE--path/to/file)}
    • returns "path/to"

echo

• (echo OPTIONS ARGS)
  • Simply echo a string as the return value of the MX
• Options
  • -w = go ahead and expand wildcards (normally this is suppressed to avoid accidental expansion since quotes are non-available within the MX)
  • --raw = do a PHP echo instead of a usual echo. This is primarily for debugging and is NOT enabled by default - you must explicitly set $EnableWikiShRawEcho to true for it to work.

fetchmail

• (fetchmail OPTIONS)
  • Get a message from a POP3 server, delete a message from the server, close the connection, etc.
  • ''Note the dependency on WikiMail version 2008-08-08 or later
• Options
  • --profile:PROFNAME = use the wikimail profile named "PROFNAME" (must be set up elsewhere)
  • --from:varname = put the value of the address from whom the email is sent in a variable named ${varname}
  • --subject:varname = put the value of the subject in a variable named ${varname}
  • --body:varname = put the value of hte body of the message in a variable named ${varname}
  • (other options such as --to:varname, etc are possible as well)
  • --stay = don't continue to the next message
  • --delete = delete the message (default if not set unless --stay is specified in which case it must be specified to actually delete it)
  • --nodelete = don't delete the message
  • --close = close the connection to the POP3 server (if you don't do this nothing will be deleted)
• Example

while fetchmail --profile:foo --from:from --subject:subject --body:body
do
   echo "====\n'''FROM: ${from}'''\n'''SUBJECT: ${subject}'''\n${body}\n"
done
fetchmail --profile:foo --close

grep (General Regular Expression Processor)

Search for a regular expression in a file.

• (grep OPTIONS PATTERN FILES)
• Options:
  • -v = reverse the sense of the search (return lines NOT matching regex)
  • -i = ignorecase (--ic or --ignorecase are synonyms) (default is to be case sensitive)
  • -l = list ONLY filenames of files that match, not the lines themselves
  • -q = don't return any lines -- just give me an appropriate return status (${STATUS}) indicating whether you found something or not (for use in if ... or while ...)
  • -F = fixed string search rather than regular expressions. Allows you to not worry about escaping "magic" characters when you aren't in need of regex capabilities.
  • --targets = search in list of target links (1 target per line) rather than in text of page
• Note: A combination of -v and -l currently acts like a -L in the "real" grep. This will be fixed to be consistent with real grep syntax in the future.

head

• (head OPTIONS FILES)
• Options:
  • -n # - (# is a number) specify how many lines of text to return from the start of a page
• Note that this is functionally equivalent to (sed -n '1,np' FILES) where the second n (inside the single quotes) is a number for n

ls (LiSt pages)

• (ls OPTIONS PAGEPATTERN ...)
  • Options
    • --os - determine all file attributes from an OS perspective instead of from a page perspective
      • size will be file size instead of number of characters on the page
      • dates will be according to file dates instead of the timestamps within the page
      • user/owner will be the UID of the owner rather than the most recent author
    • -l - give a long listing, including date, size, owner/author, etc.
    • -t - sort by date/time instead of by pagename
    • -S - sort by size instead of by pagename
    • --nosort - instead of the default sort by pagename, do no special sorting
    • -c - all dates will be create date (ctime) instead of modified date
    • -m - all dates will be modified date (time) (this is the default behavior)
    • -a - all pages will be listed - equivalent to --list:all or list=all option
    • -g - only return the group names - the period and the name will be stripped
    • -G - only return a unique list of the group names
    • -p - only return the page names - the groupname and the period will be stripped
    • -P - only return a unique list of the page names
    • --group:GROUP - identical to group=GROUP from pagelist context
    • --name:NAME - identical to name=NAME from pagelist context
  • Arguments
    • any combination of pages and files (files preceded by TEXTFILE--) can be given, using wildcards as desired
      • Any files (preceded by TEXTFILE--) will obviously always be listed with a --OS option assumed as there is no page information available
    • --list:X and list=X options (or setting of ${LIST} variable) will be honored
    • if no arguments are specified then "*.*" (all pages) is assumed
  • Simply give a list of files
  • EXAMPLE: ls -l Main.W* TEXTFILE--pmwiki.php TEXTFILE--cookbook/WikiSh.php
    • Note that the textfiles are owned by the root user and therefore have "0" as the UID

       Joe User    2643 Apr 08 08:31 Main.WikiSandbox
       Joe User     494 Jan 30 21:11 Main.WikiSh
              0  176962 Apr 14 09:13 cookbook/WikiSh.php
              0   80588 Apr 13 22:27 pmwiki.php

• EXAMPLE: ls -l --os Main.W
  • Note the difference in sizes and UID since it is from an OS level instead of from a page level

              0   40844 Apr 08 08:31 Main.WikiSandbox
              0    1617 Jan 30 21:11 Main.WikiSh

mailx (send an email)

Shell note: options for this mailx MX are largely unrelated to options for the linux mailx program. Particularly note that while stdin is accepted, unflagged arguments will be seen as pagenames rather than as addresses.
Note that mailx is dependent on WikiMail recipe. It must be installed and configured appropriately to use this MX.
Note that mailx is disabled by default. You must explicitly set $EnableWikiShMailx = true; in your config.php if you wish to use this command.

• Usage: (mailx OPTIONS PAGE)
  • Options:
    • -q - Do it quietly (don't report success/failure in the return value of the MX)
    • --to:address- send the email to this address
    • --cc:address- send the email to this address as cc
    • --bcc:address- send the email to this address as bcc
    • --subject:subj specify the subject "subj" for this email
    • --from:address specify the "from" address for this email
  • synonyms for above options
    • -t address = --to
    • -c address = --cc
    • -b address = --bcc
    • -s subject = --subject
    • -f address = --from
• Examples:
  • {(wikish echo "This is a message\nwith 2 lines" | mailx -s "nice subject" -t myaddr@mydomain.com)}
  • {(wikish echo "(:include MyGroup.MyPage:)" | mailx -s "nice subject" -t myaddr@mydomain.com)}
  • {(wikish mailx -s "nice subject" -t myaddr@mydomain.com MyGroup.MyPage MyGroup.MyPage2)}

mv (MoVe pages)

• USAGE: (mv OPTIONS PAGE1 PAGE2)
• USAGE: (mv OPTIONS PAGE1 [...] GROUP)
  • Permissions
    • Requires $EnableWikiShRemove
    • Requires $EnableWikiShRemoveFully for -f option
  • NOTE: Page history will be lost. It is NOT transferred with the moved file.
  • NOTE: Links to this page from other pages are note modified.
  • Options
    • -f DANGER - this fully deletes the source rather than just renaming it to page-del,timestamp
    • --trial - don't actually copy nor remove the files -- just produce output indicating what would have been done
  • PAGES - as usual with wildcards or specifying multiple files
  • GROUP - if multiple source pages were specified you must specify either a group name (copy those pages to the other group) or a directory for text files (TEXTFILE--dirname)
  • EXAMPLE:
    • (mv groupa.page1 groupb.page2)
      • This will rename groupa.page1 to groupb.page2
    • (mv groupa.* groupb)
      • This will move all pages in groupa into groupb
    • (mv groupa.* TEXTFILE--mydir)
      • This will place text files containing the source in a directory relative to the main pmwiki directory called mydir
• STATUS: alpha - currently just calls the {(cp ...)} and then checks the ${STATUS} and if successful then calls the {(rm ...)}. The same set of options (whatever you specify) are passed to both.
• Link solution: If you are renaming a page and want to fix the links to that page in other pages this script should do the job (beware timeouts - you may want to check a set of pages at a time rather than all pages with the *.*):
  • For a single page:

    mv GroupA.Page1 GroupB.Page2
    sed -i 's|\bGroupA[/.]\)?Page1\b|GroupB.Page2|g' (grep -l --targets '^GroupA.Page1$' *.*)
    

  • For an entire group:

    for i in GroupA.*
    do
       set -s page = `echo ${i} | sed 's/^GroupA.//'`
       mv ${i} GroupB
       sed -i 's|\bGroupA[/.]\)?{page}\b|GroupB.${page}|g' (grep -l --targets '^GroupA.${page}$' *.*)
    done
    

null

• (null OPTIONS ARGS)
  • Ummm. Do nothing. Ignore options and arguments and just swallow any output to this point.
  • Think /dev/null. Think "bit bucket."
  • May be replaced by stdout=null or >null or >/dev/null in the future.

read

• (read [OPTIONS] VAR1 [VAR2 [...]] <PAGENAME) | (read --stdin:PAGENAME VAR1 [VAR2 [...]])
  • Read in a line at a time (once per invocation) from PAGENAME. Assign each word (delimited by IFS) to subsequent variables VAR1, VAR2, etc. Automatically move on to the next line unless --nonext is set.
• Options:
  • --stdin:PAGENAME (REQUIRED! specifies where the lines will be read from) (can also be set by <PAGE syntax)
  • --IFS:regex (specify what value to split the lines by, default \s for whitespace)
  • --set:n In certain cases you may need more than 1 read going on (nested loops, for instance). In this case you will need to specify a different "set" to hold the index pointing to the current line, etc. (default is 0 when not explicitly specified)
  • --nonext don't step to the next line after reading this line - stay on the same line ("No Next")
  • --initialize read in the input file and get things ready for subsequent use of this set (this is the default if the current set is uninitialized)
  • --saveset save the current set in your session (usually because of PHP timeouts) (--save is a synonym) (no other options are honored if this one is specified)
  • --restoreset restore the current set from your session (no error if no state exists) (--restore is a synonym) (no other options are honored if this one is specified)
  • --clearset clear the current set in the session (--clear is NOT a synonym - see below!) (no other options are honored if this one is specified)
  • --status output the index to the current line - no variables are set, no increment, etc. (useful for saving a spot and returning via goto or for a progress message indicating how many lines have been processed)
  • --linecount output the number of lines in the current set - no variables are set, no increment, etc. (useful for going to the last line via goto or for a progress message indicating how far you have to go)
  • --next go to the next line (all --next alternatives which follow are relative to the current line) (this option is a shortcut for --next:1 and is the default course of action)
  • --next:n go to the nth next line
  • --next:pat go to the next line matching pat
  • --next:/pat/ go to the next line matching regex pat
  • --prev go to the previous line (all --prev alternatives which follow are relative to the current line) (shortcut for --prev:1)
  • --prev:n go to the nth previous line
  • --prev:pat go to the previous line matching pat
  • --prev:/pat/ go to the previous line matching regex pat
  • --goto:n go to the specified line (0-based, absolute addressing)
  • --rewind go to the first line (same as --goto:0)
  • --clear clear the current set (this is done implicitly when you reach the end of the file if --noclear is not specified) (if --clear is explicitly set then no other options/actions will occur - just the clear)
  • --noclear even if you get to the end of the file (or beginning of file with --prev) don't clear the set
  • --stdout:page if --stdout is specified as an option (or set via OUTPUTSPEC as in >PAGE or >>PAGE or etc) then variables are *not* loaded and the whole line which would have been split into variables is sent to stdout
• Examples:
  • {(wikish while read var1 var2 <MyPage; do; echo "Word1 is ${var1}, word2 is ${var2}"; done)}
    • For each iteration of the while loop it will read the next line from MyPage and assign the first word to ${var1} and the remainder of the line to ${var2}. The body of the loop simply prints off these values.
• Saving/Restoring in your session to deal with timeouts
  • The ability to save/clear/restore the current set in your session is provided because PHP has an annoying timeout of 30 seconds of execution time. For most PHP applications this is great, but if you are doing a search/replace of every page in your wiki (or some other time-consuming operation) then 30 seconds is probably not nearly enough time. The read command provides the capability to save where you are and pick up again later.
  • Note that the state information is saved on a set-by-set basis and so if you are using multiple sets you will need to save each one individually.
  • Note that the state information is saved in your session ($_SESSION['WIKISH_READ'] to be specific). Thus anything which causes you to lose your session or start a new session will cause this state information to be lost.
  • If multiple options are specified they are executed in this order:
    • (initialize)
    • restore
    • clear
    • (movement options)
    • save
  • This allows you to restore/clear session or restore/move/save in a single call. Note particularly that this allows you to interactively process through a file using read in the Control Panel by specifying --restore and --save on each call.
  • The typical way to run this would be like this (note comment below regarding --clearset!):

read --restoreset  # this will not cause an error if there is nothing to restore
if test -z `read --status`
then
   echo "Generating list of files to work off of..."
   ls --list:normal >Tmp
fi
while read fn <Tmp
do
   echo "Processing page ${fn}"

   if test ${SECONDSLEFT} -lt 7   # allow 7 seconds for initial compile time plus time to process next file
   then
      read --saveset
      echo "You will need to reinvoke this script to complete.  You have finished" `read --status` "files out of" `read --linecount`
      exit
   fi
done

• Note that on-screen output will *not* be saved from one invocation to the next. Thus if you are wanting to save something as a result of your processing you will need to save it in a (perhaps temporary) file via >PAGE or >>PAGE or other OUTPUTSPEC alternatives.
• Note it would probably be wise to run {(read --clearset)} BEFORE the above script just in case you have a prior readset hanging around in your session...
• It is recommended to run the above script from your control panel, but you may well want to "Suppress History/Favorites" (option towards the bottom of the page) while you are re-invoking the script -- that way you will maximize the time spent on processing files and minimize the time spent processing history file...

rm

• (rm OPTIONS FILES)
• Permissions
  • Requires $EnableWikiShRemove
  • Requires $EnableWikiShRemoveFully for -f options
• Options:
  • -f - completely delete the files (no Group.Page,del-1234567 "backup" will remain)
  • --trial - don't actually delete any files -- just produce the usual output to tell me what you would have done
• Note that without the -f option the file will be deleted by writing the word "delete" into the page content and then calling UpdatePage(). With the -f option that process will be done first (to hopefully clean up any page caching and etc.) and then the resulting FILE,delTIMESTAMP file will be deleted.
• Note that the -f is still causing problems...
• Note that if $EnableWikiShRemove is not set to true then (rm ...) will do NOTHING
• Note that $EnableWikiShRemoveFully must be set in order for the -f flag to be noticed and acted upon.

sed (StreamEDitor)

• (sed OPTIONS EDIT-ACTION FILES)

OR (to specify multiple edit-actions)

• (sed OPTIONS -e EDIT-ACTION [-e EDIT-ACTION ...] FILES)

Currently only 2 edit-actions are implemented:

• p=print. FORM: address1,address2p
  • (sed -n 'address1,address2p' file ...)
  • EXAMPLE: (sed -n '2,5p' group.file)
  • EXAMPLE: (sed -n '/pat1/,/pat2/p' group.f*)
  • each address can be either a line number (\$ = last line of file - note the \ before the $-sign) or a /regular expression/ (intermix them freely)
  • the "-n" flag is necessary to prevent automatic printing of ALL lines
• s=substitute. FORM: s/searchfor/replace/flags
  • (sed 's/searchfor/replace/g' FILES)
  • flags:
    • g = global search/replace -- if you don't specify this then only the first occurrence on a line is replaced
    • F = fixed string search/replace -- no regular expressions so no "magic" characters - no need to escape the world if you aren't using regexes.
      • Basically uses str_replace() instead of preg_replace(). Since str_replace() has no way to limit the number of replacements a flag 'F' requires a flag 'g' and will generate an error message without it.
    • i = case Insensitive search
    • A = anchored to the beginning (no need for ^)
    • D = anchored to the end (no need for $)
    • U = Ungreedy regex
    • X = implements PCRE_EXTRA (see PCRE pattern modifiers)
    • u = pattern strings treated as UTF-8
      • Note that WikiSh makes no claims on being UTF-8-tested -- I just saw the feature was available here and allowed it to pass through to preg_replace -- use at your own risk.
  • search/replace specification
    • s is required as the command
    • the forward slashes delimiting the search and the replace can be other characters as well - the following are all equivalent:
      • s/search/replace/g
      • s|search|replace|g
      • s^search^replace^g
  • EXAMPLE: (sed 's/myname/yourname/g' Profiles.*)
• OPTIONS
  • -i=in-place editing. This gives you capability of doing a search-and-replace on a file or set of files and write the results back to the files. (Note that no write will be made if no change is made. If you are doing search/replace and you don't find the search string then the page will not be written.)
    • -v=verbose. If you are doing in-place editing and specify -v then an output of "Updated: PAGENAME" will be generated when a change is made to a file.
  • -n=don't print ALL lines but only those specified by the "p" command. (this is necessary for the "p" command -- otherwise you say 1,2p but since all lines are printed there's no differentiation between 1-2 that you asked to be printed and all the rest)
• FUTURE edit-commands: a=append, d=delete

You can specify multiple edit-actions with multiple uses of the -e flag:

• {(sed -e '1,2p' -e '/#STARTSECTION/,/#ENDSECTION/p' -e 's/not-nice-word/nice-word/g' mygroup.myfile)}

set (set variables for use within WikiSh MX)

• Usage: (set OPTIONS VAR ASSIGN-OP EXRESSION)
  • Options:
    • -v - Return the value assigned (be Verbose)
    • -s - Process expression as a String. Simply place quotes around arguments after the assignment operator. Required for string assignment.
    • --pv - Set it as a page variable instead of a WikiSh Variable. (Normally requires invoking via {earlymx(...)} instead of the standard {(...)} if you are going to use the PV.
    • --session - Set it as a SESSION variable instead of a WikiSh Variable. This makes it "stick" between loading of different pages.
  • VAR - any character sequence that will match "/\w+/" (note that you do NOT prepend the variable name with a dollar-sign. Use i instead of ${i}. See section on "Variables" above.)
  • ASSIGN-OP - normally simply =, but it can be anything recognized by php (=, +=, -=, .=, etc.)
  • EXPR - a valid PHP expression (can contain PV, PTV, WikiSh variables, etc.)
• NOTE: There's not much error-checking going on here -- it's your responsibility to make sure you get a valid expression going... I do enforce certain valid functions and don't allow others to be used (too big a security risk otherwise).
• Examples:
  • {(set i ++)} - increment ${i} by 1
  • {(set i += 1)} - increment ${i} by 1
  • {(set i = (wc -l main.pagename))} - set ${i} to the number of lines in the page main.pagename
• See section above on variables for more details

sleep

• (sleep n)

This makes your script "go to sleep" for n seconds.

• Options:
  • no options at this point
• Arguments:
  • n specify the number of seconds to sleep (n is a number [integer or float] and is a required argument)

sort

• (sort OPTIONS FILES OUTPUTSPEC)
• Options:
  • -u (unique sort - return a list of unique lines as defined by -k keys)
  • -d or --dictionary-order (ignore all characters except letters, numbers, and whitespace when sorting)
  • -b or --ignore-leading-blanks (ignore leading whitespace when sorting)
  • -f or --ignore-case (ignore case when sorting)
  • -r or --reverse (sort in descending order)
  • -t X or --field-separator:X (use X as a field separator (X is a single character)
  • -k POS[,POS][OPTS] or --key:POS,POS[OPTS] (specify which key (separated by field-separator above and 1-based) to sort on)
    • This option is repeatable as often as you want
    • Each POS takes the form FIELD[.CHAR]
    • The first POS indicates a starting point and the second POS indicates an ending point.
    • If no second POS is given then the sort will consider the remainder of the line significant in the sort
    • Any of the d/b/f can be specified on a per-key basis by appending it to the end of the key specifier
• NON-SHELL Options:
  • --count (print a prefix of the count of duplicates of that line)
  • --duplicated (print only lines which have a duplicate)
  • --only-unique (print only lines which are unique - do not print lines that are duplicated)
  • --no-sort (don't do the sort -- do everything else but don't actually sort)
• Examples:
  • {(sort -k 2,2 Main.*)} - sort all lines in all pages in Main. Use whitespace as the field separator and sort only on the 2nd field
  • {(sort -t: -k 2,3d Site.Passwd)} - sort lines in Site.Passwd. Use colon to separate fields and sort on fields 2 and 3 using dictionary sort order.

tail

• (tail OPTIONS FILES)
• Options:
  • -n # - (# is a number) specify how many lines of text to return from the end of a page
• Note that this is functionally equivalent to (sed -n '-n,$p' FILES) where the 2nd n (inside the single quotes) is a number for n.

test

• (test ARGS/OPTS)
• Options & Arguments:
  • FILE OPTIONS:
    • -d FILE (is a directory/group)
    • -e FILE (exists)
    • -f FILE (exists)
    • -g GROUP (does it exist as a group) (non-shell)
    • -r FILE (is readable)
    • -s FILE (file has something - is non-zero length)
    • -w FILE (is writable)
    • FILE1 -nt FILE2 is file1 newer than file2 by modification date
    • FILE1 -ot FILE2 is file1 older than file2 by modification date
  • STRING OPTIONS
    • -z STRING (true if string is zero length)
    • -n STRING (true if string is non-zero length)
    • string1 == string2
    • string1 != string2
    • string1 ~= /regex/
    • string1 < string2
    • string1 > string2
  • INTEGER OPTIONS
    • INT1 -eq INT2
    • INT1 -ne INT2
    • INT1 -lt INT2
    • INT1 -le INT2
    • INT1 -gt INT2
    • INT1 -ge INT2

uniq

• (uniq OPTIONS FILES OUTPUTSPEC)
  • NOTE: uniq does NOT follow shell usage for specifying fields and characters. Instead it uses the -kPOS1,POS2 syntax from sort. The -k syntax is more flexible and powerful.
  • NOTE: uniq identifies duplicates if they are next to each other. If the input is unsorted then duplicates will not be identified.
• Options:
  • -d (only print duplicate lines)
  • -c (print the count of duplicates before each line)
  • -f NOT IMPLEMENTED (use -k syntax instead)
  • -s NOT IMPLEMENTED (use -k syntax instead)
  • -w NOT IMPLEMENTED (use -k syntax instead)
  • -u (print only lines that are unique - do not print lines that are duplicated)
  • -k POS[,POS][OPTS] or --key:POS,POS[OPTS] (specify which key (separated by field-separator above and 1-based) to sort on)
    • This option is repeatable as often as you want
    • Each POS takes the form FIELD[.CHAR]
    • The first POS indicates a starting point and the second POS indicates an ending point.
    • If no second POS is given then the sort will consider the remainder of the line significant in the sort
• Examples:

wc (WordCount)

• (wc OPTIONS FILES)
  • OPTIONS
    • -l = count lines
    • -w = count words
    • -c = count characters
    • -q = quiet - don't print labels "line:"/"words:"/"characters:" before the actual counts
    • Any combination of the above works as well. Note that not specifying ANY of those defaults to ALL of them.

Non-shell commands

wikish_button

• (wikish_button [-q] ID [label])
  • wikish_button simply activates or deactivates WikiSh in general (see wikish_active below for more details)
  • ARGUMENTS
    • ID - an identifier for the button or set of actions (same ID can be used for other "buttons" to make them act in concert with one another, normally with the -q option all but one)
    • LABEL - optional label on the button. If not specified the "ID" will be used as the label.
  • OPTIONS
    • -q = don't actually display a button in this position -- just handle the activating/deactivating based on a button somewhere else on the same page with the same ID.
  • Examples
    • {(wikish_button a "List Files")(ls mygroup.*)}
      • This will list files in mygroup after the button labelled "List Files" is pressed
    • {(wikish_button -q a)(ls yourgroup.*)}
      • If on the same page as the previous example, this will list all pages in yourgroup when the above button is pressed. No button will be generated in THIS location, but the ls command will be activated/deactivated appropriately.

wikish_controlpanel

• (:wikish_controlpanel [method]:)
  • Explanation of features
    • When this markup is placed on a page it will display a form allowing you to type in WikiSh commands, execute them, see the results, have the executed commands saved in a history file, execute commands from history, have the option to mark some of those history commands as "favorites", and then execute commands from the list of favorites as well.
  • Appearance
    • The form itself is simply a large textarea for typing in the commands, buttons underneath for executing the commands and clearing the command, and then below that a selectbox on the left containing history and a selectbox on the right containing favorites. Underneath each selectbox are buttons to execute commands, copy back and forth, etc.
  • Notes on usage
    • When typing in commands you do NOT type {(command)} -- you simply type the command. In other words if you want to do {(echo abc)} you just type "echo abc" in the command box.
    • Whatever you type will be placed in a {(wikish ...)} context, so if/then/else/fi, while/do/done, for var in list/do/done, source, etc. are all valid commands to type.
    • As in {(wikish ...)} you can separate commands with semi-colons.
    • Unlike {(wikish ...)} you can also put commands on multipl