ls *.mp3 | xargs rm
ls *.mp3 | xargs -i cp {} /home/bla
ls *.mp3 -d -> stays in current dir and doesn't enter mp3/
```
- start a restricted shell (no cd, etc) to test programs
`# bash -r`
- start shell with options
`# bash -O [modification]`
`autocd -> use the argument as cd command`
`cd-spell -> corrects dir names`
`dirspell -> corrects dir names in file completion`
`no-caseglob -> case insensitive`
---
to run commands:
`# command1 && command2`
logical AND :: run command1, if ok then run command2
`# command1 || command2`
logical OR :: run command2 only if command1 fails
# cp
`cp -rv `
gives feedback
`cp -p`
preserves original date/time
# dd
use blocks of 64k and report every 10Mb:
`dd if=/dev/hda | buffer -s 64k -S 10m > image`
or, from other console:
`kill -SIGUSR1 $(pidof dd)`
# df
`df -type=ext3,ReiserFS`
`df -H human readable`
# find
find all files in your homedir modified or created today
`find ~ -type f -mtime 0`
case insentive search
`find . -iname "*mp3"`
find files and directories in yor homedir not created by you
`find ~ ! -user ${USER}`
and fix this
`find ~ ! -user ${USER} -exec sudo chown ${USER}:"{}" \;`
find duplicates PvdM
`find . -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 15`
# free
`free -m -> megabytes`
`free -t -> totals`
`free -s[seconds] -> update interval`
watch the line with +/- buffers: **free** indicates memory that can be freed instantaniously **used** is all memory being used
# grep
`grep -A2 bla greps 2 lines after`
`grep -B3 bla greps 3 lines before`
# history
`# history -d [postition] -> remove specific command`
`# history -c -> clear entire history`
# iotop
monitors io proces
`iotop -o makes output more readable by showing only processes causing io right now `
`iotop -d10 refresh interval`
`iotop -o -b -d10 -n30 > io.txt batch mode: write io status 30 times to file at 10s interval`
# iperf
`iperf -s -w128k run server for performance benchmark`
`iperf -c `` -w128k -t30 -r run client`
`iperf -c `` -w128k -t30 -d run client in two directions, full duplex`
# kill
kill -l shows instructions
` 1 SIGHUP restart immediately after terminating, or triggers reconfiguration of background service`
` 2 SIGINT Ctrl+c from the keyboard, terminate!`
` 3 SIGQUIT Ctrl+\ from the keyboard, terminate with coredump`
` 9 SIGKILL force terminate, extreme signal that can't ignored`
`11 SIGSEGV program attempted an invalid memory reference, terminate with coredump`
`15 SIGTERM request to terminate and cleanup`
`19 SIGSTOP interrupts the process until you enter SIGCONT to continue`
`kill -19 9102 10234 stops several processes`
`killall -19 ssh sends all ssh connections to sleep`
`killall -19 -i ssh interactive mode (recommended)`
# lsof
`lsof list open files`
`lsof -i :22 list all ssh connections (same :ssh)`
`lsof -i@10.0.0.1 list all connections from 10.0.0.1`
`lsof -i@amsterdam.nl list all connections from domain`
`lsof -u username list all open files from username`
`lsof -c bash list open files related to bash`
`lsof -c /log/ list all open files from all processes containing log`
# lspci
`lspci -v `
`lspci -vv`
`lspci -vvv`
`lspci -nn -> show names & numerical `
`lspci -k -> show kernel modules associated with the device`
# more or less
`+`` number of line from which to start displaying`
`z `` number of lines to jump forward`
`y `` number of lines to jump back`
`-I ignore case when searching`
`-V underline tabs, line endings`
`/`` highlights all ocurrences of search`
# pgrep
`pgrep ssh lists all ssh processes`
`pgrep -l ssh shows names`
`pgrep -f shows full commandline`
`pgrep -u show user`
# ping
`ping 0 -> pings localhost`
`ping c 5 nu.nl -> pings 5 times`
`ping -f host -> floods the host`
`ping -a IP -> gives audible beep Note: It can give beep only from terminal number 1 through 7 and gnome-terminal ( It will not work in console ).`
`ping -q ip -> show only summary`
`ping -s 100 -> change packet size`
`Pressing CTRL+| (Control key followed by pipe symbol) for the shows the summary in between, and continues with it packet sending and receiving process.`
`ping hop1 hop2 hop3 .. hopN destination`
`ping -R 192.168.1.63 -> Record and print route of how ECHO_REQUEST sent and ECHO_REPLY received`
# pkill
understands the same options as pgrep
`pkill -19 ssh`
`pkill -19 -u joop stop joop's processes`
# ps
show how long a process is running
`ps -o pid,etime,cmd `pidof amarok``
show memory usage per process
`ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS`
# pstree
`pstree -a shows processes tree structure with program's parameters`
`pstree -h highlights own process `
`pstree -H `` highlights process id`
`pstree -u shows user`
# stat
shows status of file
`stat /etc/passwd `
`stat -c "%x %n" /etc/passwd formatted to show time`
# swapon
`swapon -s see size and usage`
# tail or head
`--retry keep monitoring the file `
`-s seconds how often monitoring retry`
`-pid `` stops tail from running if PID stops running`
# tar
`tar zxf `` -> extracts file`
# top
```
PID programma id
USER gebruiker
PR prioriteit van de taak
NI nice value
VIRT virtuele geheugen gebruik
RES fysieke geheugen gebruik
SHR deel v/h gebeugen dat mogelijk met andere taken wordt gedeeld
%CPU percentage processorgebruik
%MEM percentage geheugengebruik
TIME+ de tijd dat de taak op de cpu heeft gedraaid
COMMAND de naam van de taak
```
keys: z: enable color mode. x: show sorted column in different color < >: change sorted column u \[user\]: show only users processes d: delay in sec
show only user user
`#top -u user`
# zombies
`ps aux | grep defunct`
# AutoYast
create autoyast file through icon in Yast2->autoyast
`www.suse.de/~ug (Uwe Gansert)`
# Midnight Commander (MC)
The standard colorpallette is incredibly hard to read in certain terminals, like yakuake.
For readable colors (mostly black and green), enter in your ~/.mc/ini file the following line:
`[Colors]`
`base_color=lightgray,green:normal=green,default:selected=white,gray:marked=yellow,default:markselect=yellow,gray:directory=blue,default:executable=brightgreen,default:link=cyan,default:device=brightmagenta,default:special=lightgray,default:errors=red,default:reverse=green,default:gauge=green,default:input=white,gray:dnormal=green,gray:dfocus=brightgreen,gray:dhotnormal=cyan,gray:dhotfocus=brightcyan,gray:menu=green,default:menuhot=cyan,default:menusel=green,gray:menuhotsel=cyan,default:helpnormal=cyan,default:editnormal=green,default:editbold=blue,default:editmarked=gray,blue:stalelink=red,default`
# tune kernelparameters
stored in /proc/sys/ gone after reboot
sysctl is a service that reads sysctl.conf during boot and read parameters from there -> makes it permanent
`# ulimit -a`
givesinfo about system resources limitations
# Using x2x to connect two computers and their screens with one mouse and keyboard
Let's say you have 2 computers:
`cpu1: 10.0.0.1`
`cpu2: 10.0.0.2`
Let's say cpu1's screen is left to cpu2's screen.
Let's say you want to control cpu1 from cpu2.
Let's assume both computers have the same username.
1. install x2x on cpu1
2. enter on cpu2
`ssh -X user@cpu1 'x2x -west -to :0'`
and leave terminal open.
And that's it!
# Script to check and restart a process
```
#!/bin/bash
# -------------------------------------------------------------------------
# Copyright (c) 2003 nixCraft project
# This script is licensed under GNU GPL version 2.0 or above
# -------------------------------------------------------------------------
# This script is part of nixCraft shell script http://en.wikipedia.org/wiki/Shell_script collection (NSSC)
# Visit http://bash.cyberciti.biz/ for more information.
# Copyright (c) 2010 CyberOrg Info. Modification to watch any process/service
# -------------------------------------------------------------------------
RESTART="/etc/init.d/sshd restart"
PGREP="/usr/bin/pgrep"
processtowatch="sshd"
x=1
while [ $x = 1 ];
do
# find pid
$PGREP ${processtowatch}
if [ $? -ne 0 ]
then
echo "`date` : $processtowatch went down, restarting" >> /var/log/restart.log
$RESTART
fi
sleep 30
done
```
| ***Key*(s):**
| ***Movement*:**
|
| 0 (zero)
| 1stcolumn on the line
|
| ^
| 1st**non-blanc**char on line
|
| $
| Last char on line
|
| g\_
| Last **non-blanc** char on line
|
| w
| Next word
|
| W
| Next word and **skip** interpunction
|
| e
| End of current word
|
| E
| End of current word **incl** .interpunction
|
| b
| Begin of current word
|
| B
| Begin of current word **incl**. interpuction
|
| ge
| End of previous word
|
| gE
| End of previous word and **skip** interpunction
|
| O
| Insert new line and switch to insert mode
|
| |
| ***Key*(s):**
| ***Action*:**
|
| x
| Delete char **under** cursor
|
| X
| Delete char **before** cursor
|
| r
| Replace char under cursor with next typed char
|
| |
| d
| Delete chars until position of next command
|
| dd
| Delete line
|
| y
| Copy text until position of next command
|
| yy
| Copy line
|
| p
| Paste text **after**cursor
|
| P
| Paste text **before**cursor
|
| |
| ***Key(s):***
| ***Selection:***
|
| V
| Start selecting block of text
|
| x
| Cut block of text
|
| c
| Copy block of text
|
| iw
| Word
|
| aw
| Word **incl** spaces
|
| is
| Sentence
|
| as
| Sentence **incl** spaces
|
| ip
| Paragraph
|
| ap
| Paragraph **incl** spaces and **empty** lines
|
| i(
| Block between brackets
|
| a(
| Block **incl** brackets
|
| i<
| Block between < >
|
| a<
| Block **incl** < >
|
| i{
| Block between { }
|
| a{
| Block **incl** { }
|
| i”
| Block between “ “
|
| a”
| Block **incl**“ “
|
| i`
| Block between ` `
|
| a`
| Block **incl**` `
|
| |
| ***Search:***
| ***Action:***
|
| /text
| Search for text starting at cursor
|
| n
| Find next
|
| N
| Find previous
|
| //
| Repeat last search
|
| \*
| Search forward for word currently under cursor
|
| \#
| Search backwards for word currently under cursor
|
| |
| :set hlsearch
| Highlights all matches
|
| :set nohlsearch
| Disable ^
|
| :nohl
| Don't show highlights from last search
|
| :set incsearch
| Enables incremental search
|
| :set noincsearch
| Disables ^
|
| |
u undo
cut multiple lines: ###yy
delete rest of line from cursor: SHIFT-d
## To open multiple files on a window
- Horizontal split
`vi -o file-one file-two`
- Vertical split
`vi -O file-one file-two`
- Switch between the panes you are using.
` CTRL + W + Left key to activate left windows`
` CTRL + W + Right key to activate right windows`
` CTRL + W + Up key> to activate to windows above current one`
` CTRL + W + Down key> to activate to windows down current one`
- Split Open one file at a time
`vi file-one`
- To open the second file, go to the command mode (Esc)
`:new file-two`
OR
`:split file-two`
- to split vertically
`:vsplit`
- Typ 'CTRL-w w' to switch between the windows
- Open new file:
`:e newfile`
### Tabs
To open Vi with multiple tabs open:
`vi -p /var/log/*.log`
- switch between tabs with *gt*
- open new tab
`:tabnew`
`or`
`:tabe file`
to open new file in new tab
## Search and Replace
Go in command mode and type:
Replace First occurence of string
`:%s/string-to-replace/replace-with-string/`
Replace All occurence of string
`:%s/string-to-replace/replace-with-string/g`
Replace All occurence of string with confirmation
`:%s/string-to-replace/replace-with-string/gc`
### avoid staircase effect when pasting
If you're working in Vim and paste something into the terminal, sometimes you'll get a "staircase" effect where each line is progressively spaced farther outward, like so:
```
line 1
line 2
line 3
line 4
```
Obviously, this isn't usually desirable.
To correct this, you can enable paste to prevent the staircase effect. In command mode, type:
`:set paste`
This isn't on by default. When paste is enabled, it disables mapping and some other functions, so you probably want to know how to turn it off as well:
`:set nopaste`
## Highlighting
syntax highlighting for programming languages
`:syntax on`
# Store settings
Put your most used commands, like *:syntax on* in ~/.vimrc
# statusline
`set statusline=%F%m%r%h%w%=\ [%Y]\ [%{&ff}]\ [%04l,%04v]\ [%p%%]\ [%L]`
in /etc/vimrc
# Verhuis een linux systeem naar nieuwe hardware
Verhuis van systeem A naar systeem B.
# Inpakken
- boot systeem A vanaf liveCD
- (alternatief: boot in single-user-mode)
- mount de rootpartitie van systeem A:
`mount /dev/sda1 /mnt`
- mount het medium waarop het archief komt. Vaak automatisch, anders:
`mount /dev/sdf1 /media`
- ga naar root van (gemounte) filesysteem
`cd /mnt`
- (als je gebruikt maakt van de single-user-mode, dan cd / uiteraard)
- pak in:
`tar --one-file-system -cpf /media/systeem.tar * (-j=compressie)`
# Uitpakken
- boot systeem B vanaf liveCD
- maak nieuwe disk aan, en partitioneer. Bijvoorbeeld:
`fdisk /dev/sda`
- denk er ook aan om een swap partitie aan te maken (type 82)
- maak bestandssystemen aan:
`mkfs.ext3 /dev/sda1`
`mkswap /dev/sda2`
`etc....`
- mount de doelpartitie en de gegevensdrager:
`mount /dev/sda1 /mnt`
`mount /dev/sdf1 /media`
- pak het systeem uit:
`cd /mnt`
`tar -xf /media/systeem.tar (-j als je compressie gebruikt)`
# Inrichten
- pas /etc/fstab aan, vervang bijvoorbeeld UUID's door apparaatnamen zoals /dev/sda1
`/dev/sda1 / ext3 acl,user_xattr 1 1`
`/dev/sda2 swap swap defaults 0 0`
- verwijder 2 udev regels voor netwerkkaarten en optische drives:
`rm /mnt/etc/udev/rules.d/70-persistent-cd.rules`
`rm /mnt/etc/udev/rules.d/70-persistent-net.rules`
- deze worden na booten opnieuw aangemaakt.
- pas /mnt/etc/X11/xorg.conf aan of wis deze zodat deze opnieuw wordt aangemaakt.
- pas GRUB aan:
- pas de 'root=' regel aan
- vervang in /mnt/boot/grub/menu.lst UUID's door apparaatnamen zoals /dev/sda1
- mount nu de proc en dev bestandssystemen:
`mount -t proc none /mnt/proc`
`mount -o bind /dev /mnt/dev`
- ga dan met chroot naar het nieuwe systeem:
`chroot /mnt`
- en installeer grub bootloader:
`grub-install /dev/sda (exit met CRTL-D)`
# A guide to using systemd
This is how to stop a running service temporarily:
`systemctl stop servicename.service`
Send a kill:
`systemctl kill sshd.service`
`systemctl kill -s HUP sshd.service`
This stops it from starting at boot, but does not stop a running service:
`systemctl disable servicename.service`
And there is one way to really really stop a service for good, short of uninstalling it, and that is masking it by linking it to /dev/null:
`ln -s /dev/null /etc/systemd/system/servicename.service`
`systemctl daemon-reload`
When you do this you can't even start the service manually. Nothing can touch it.
What if you change your mind? Pish tosh, it's easy. Simply delete the symlink and run
`systemctl enable servicename.service.`
While we're here, let's talk about two reload commands: daemon-reload and reload. The daemon-reload option reloads the entire systemd manager configuration without disrupting active services. reload reloads the configuration files for specific services without disrupting service, like this:
`systemctl reload servicename.service`
This reloads the actual configuration file used by the hardy sysadmin, for example the /etc/ssh/sshd\_config file for an SSH server, and not its systemd unit file, sshd.service. So this is what to use when you make configuration changes.
List active service units:
`systemctl --type=service`
list all service units:
`systemctl --type=service --all`
list systemd services:
`systemctl list-unit-files`
# default target
The system boots to **default.target**
This is a symbolic link to **multiuser.target** or **graphical.target**.
Changing the default runlevel/target is replacing the symbolic link
`ln -sf /lib/systemd/system/multi-user.target /etc/systemd/system/default.target`
To change runlevel:
`systemctl isolate multi-user.target`
# booting
At boot time, on the kernel grub line, add:
**`systemd.unit=xxxxx`**
So what's taking so long? We can find out with the systemd-analyze blame command
```
$ systemd-analyze blame
60057ms sendmail.service
51241ms firstboot-graphical.service
3574ms sshd-keygen.service
3439ms NetworkManager.service
3101ms udev-settle.service
3025ms netfs.service
2411ms iptables.service
2411ms ip6tables.service
2173ms abrtd.service
2149ms nfs-idmap.service
2116ms systemd-logind.service
2097ms avahi-daemon.service
1337ms iscsi.service
```
If you like pretty graphs systemd includes a cool command for automatically generating an SVG image from the blame output, like this:
`systemd-analyze plot > graph1.svg`
To shutdown:
`systemctl poweroff`
# systemctl files
`/lib/systemd/system/ `
`/etc/systemd/system/ is for customized settings`
# Perform bechmarks on the filesystem
- To test the speed of the filesystem (not the physical disk!) you can use the following script:
```
#!/bin/bash
# USAGE:
# ./speed_test.sh /path/to/my/file /path/to/destination number_of_tests
NUM_TESTs=$3
SUM=0
for i in $( seq 1 $NUM_TESTs ); do
REC=`dd if=$1 of=$2 2> some_random_file_ ; cat some_random_file_ | cut -d " " -f8 | tail -1`
SUM=`echo $SUM + $REC | bc`
done
RESULT=`echo $SUM / $NUM_TESTs | bc | awk '{ str1=str1 $0 }END{ print str1 }'`
echo $RESULT MB/s
#clean up
rm some_random_file_
rm $2
```
```
def quickSort(arr):
less = []
pivotList = []
more = []
if len(arr) <= 1:
return arr
else:
```
```
def quickSort(arr):
less = []
pivotList = []
more = []
if len(arr) <= 1:
return arr
else:
```
| NF
| number of fields in the current line
|
| NR
| the current record number (line number)
|
| FS
| input field seperator (space by default)
|
| RS
| record seperator (newline by default)
|
| Example
| Example
|
| Example
| Example
|
| Example
| Example
|
| Example
| Example
|
| Example
| Example
|
show the 5th record on the second line:
`df /tmp | awk 'NR==2 {print$5 }'`
## use awk with a program
the file **maxuid**:
```
BEGIN {maxuid = 0; FS=":" }
{ if ($3 > maxuid) maxuid = $3 }
END { print" the largest UID is ", maxuid }
```
then use awk like this:
`$ awk -f maxuid /etc/passwd`
to ignore user nobody:
```
BEGIN {maxuid = 0; FS=":" }
$1 != "nobody" { if ($3 > maxuid) maxuid = $3 }
END { print" the largest UID is ", maxuid }
```
cat -n: (adds line numbers)
`$ awk '{ print NR, $0}' /etc/fstab`
wc -w: (count words)
`$ awk '{ w += NF } END { print w}' /etc/fstab`
grep chris /etc/passwd:
`$ awk '/^chris/' /etc/passwd`
## writing self contained scripts
```
#!/usr/bin/awk -f
{ cost[$1] += $2*$4 }
END { for (cat in cost) print cat, cost[cat] }
```
`chmod u+x catscript`
`./catscript `
[http://www.pement.org/awk/awk1line.txt](http://www.pement.org/awk/awk1line.txt)
# Systemd journal log file setting
The new openSUSE 13.2 journald (from systemd) is sometimes quickly using up disk space.
There are setting you can tweak in /etc/systemd/journald.conf
```
[Journal]
Compress=yes
SystemMaxUse=500M
SystemKeepFree=2G
```
After that, restart the services **a few times** for the changes to take effect:
`systemctl restart systemd-journald.service `
`systemctl restart systemd-journal-flush.service `
You should see a reduction in the size of the /var/log/ directory almost immediately.
# Dealing with spaces in filenames
BASH for loop works nicely under UNIX / Linux / Windows and OS X while working on set of files. However, if you try to process a for loop on file name with spaces in them you are going to have some problem. For loop uses $IFS variable to determine what the field separators are. By default $IFS is set to the space character. There are multiple solutions to this problem.
# Set $IFS variable
Try it as follows:
```
#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
for f in *
do
echo "$f"
done
IFS=$SAVEIFS
```
OR
```
#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
# set me
FILES=/data/*
for f in $FILES
do
echo "$f"
done
# restore $IFS
IFS=$SAVEIFS
```
# More examples using $IFS and while loop
Now you know that if the field delimiters are not whitespace, you can set IFS. For example, while loop can be used to get all fields from /etc/passwd file:
```
....
while IFS=: read userName passWord userID groupID geCos homeDir userShell
do
echo "$userName -> $homeDir"
done < /etc/passwd
```
source: [http://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html](http://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html)
# View progress of dd
In Linux, find running dd processes and send them a signal to print out their progress.
`kill -USR1 $( pidof dd )`
# Sort installed rpms by size
Sort installed RPMs by size
To list all RPM packages installed on a Linux machine sorted from the largest to the smallest in size:
`rpm -qa --queryformat '%{size} %{name}\n' | sort -rn | more`
# Command line tips
Show all VMs:
`vboxmanage list -l vms`
Show the state of the VM:
`vboxmanage showvminfo w7new | grep State`
Control the state of the VM:
`vboxmanage controlvm w7new pause|resume|reset|poweroff|savestate|`
` acpipowerbutton|acpisleepbutton`
# Find
- [find latest files in all subdirectories per directory](https://www.reeltoreel.nl/knowledgebase/books/homelab/page/find-latest-files-in-all-subdirectories-per-directory)
# Find latest files in all subdirectories per directory
Create a file latestfile.sh:
```
#!/bin/bash
ls -lrt "$1" | tail -n1
```
_#_Bash_/_Ksh_match=b*r_[[_$foo_==_"$match" "wikilink") # Good! Unquoted would also match against the pattern b\*r.
You may have seen code like this:
```
# POSIX / Bourne
[ x"$foo" = xbar ] # Ok, but usually unnecessary.
```
The `x"$foo"` hack is required for code that must run on *very* ancient shells which lack \[\[BashFAQ/031|[](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/), and have a more primitive `\[`, which gets confused if `$foo` begins with a `-`. On said older systems, `\[` still doesn't care whether the token on the right hand side of the `=` begins with a `-`. It just uses it literally. It's just the left-hand side that needs extra caution.
Note that shells that require this workaround are not POSIX-conforming. Even the Heirloom Bourne shell doesn't require this (probably the non-POSIX Bourne shell clone that's still most widely in use as a system shell). Such extreme portability is rarely a requirement and makes your code less readable (and uglier).
<<Anchor(pf5)>>
## cd $(dirname "$f")
This is yet another [quoting](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes)error. As with a variable expansion, the result of a CommandSubstitution undergoes WordSplitting and [pathname expansion](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob). So you should quote it:
```
cd -P -- "$(dirname -- "$f")"
```
What's not obvious here is how the [quotes](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes)nest. A C programmer reading this would expect the first and second double-quotes to be grouped together; and then the third and fourth. But that's not the case in Bash. Bash treats the double-quotes *inside* the command substitution as one pair, and the double-quotes *outside* the substitution as another pair.
Another way of writing this: the parser treats the command substitution as a "nesting level", and the quotes inside it are separate from the quotes outside it.
<<Anchor(pf6)>>
== \[ "$foo" = bar && "$bar" = foo \] == You can't use `&&` inside the [old test (or \[) command](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq031). The Bash parser sees `&&` outside of `[](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/)` or `(( ))` and breaks your command into *two* commands, before and after the `&&`. Use one of these instead:
```
[ bar = "$foo" ] && [ foo = "$bar" ] # Right! (POSIX)
[[ $foo = bar && $bar = foo ]] # Also right! (Bash / Ksh)
```
(Note that we reversed the constant and the variable inside `\[` for the legacy reasons discussed in pitfall #4. We could also have reversed the `\[\[` case, but the expansions would require quoting to prevent interpretation as a pattern.) The same thing applies to `||`. Either use `\[\[` instead, or use two `\[` commands.
Avoid this:
```
[ bar = "$foo" -a foo = "$bar" ] # Not portable.
```
The binary `-a` and `-o`, and `(` / `)` (grouping) operators are XSI extensions to the POSIX standard. All are marked as obsolescent in POSIX-2008. They should not be used in new code. One of the practical problems with `\[ A = B -a C = D \]` (or `-o`) is that [POSIX does not specify](http://www.opengroup.org/onlinepubs/9699919799/utilities/test.html "wikilink") the results of a `test` or `\[` command with more than 4 arguments. It probably works in most shells, but you can't count on it. If you have to write for POSIX shells, then you should use two `test` or `\[` commands separated by a `&&` operator instead.
<<Anchor(pf7)>>
## [$foo > 7]($foo_>_7 "wikilink")
There are multiple issues here. First, the \[\[BashFAQ/031|[command](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/command)should *not* be used solely for evaluating [arithmetic expressions](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/arithmeticexpression). It should be used for test expressions involving one of the supported test operators. Though technically you *can* do math using some of `[`'s operators, it only makes sense to do so in conjunction with one of the non-math test operators somewhere in the expression. If you just want to do a numeric comparison (or any other shell arithmetic), it is much better to just use `(( ))` instead: <pre> # Bash / Ksh ((foo > 7)) # Right! \[\[ foo -gt 7](`'s_operators,_it_only_makes_sense_to_do_so_in_conjunction_with_one_of_the_non-math_test_operators_somewhere_in_the_expression._If_you_just_want_to_do_a_numeric_comparison_(or_any_other_shell_arithmetic),_it_is_much_better_to_just_use_`((_))`_instead:__#_Bash_/_Ksh_((foo_>_7))_#_Right!_[[_foo_-gt_7 "wikilink") # Works, but is pointless. Most will consider it wrong. Use ((...)) or let instead.
If you use the `>` operator inside `[](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/)`, it's treated as a string comparison (test for collation order by locale), *not* an integer comparison. This may work sometimes, but it will fail when you least expect it. If you use `>` inside `\[ \]`, it's even worse: it's an output redirection. You'll get a file named `7` in your directory, and the test will succeed as long as `$foo` is not empty.
If strict POSIX-conformance is a requirement, and `((` is not available, then the correct alternative using old-style `\[` is
```
# POSIX
[ "$foo" -gt 7 ] # Also right!
[ $((foo > 7)) -ne 0 ] # POSIX-compatible equivalent to ((, for more general math operations.
```
Note that the `test ... -gt` command will fail in interesting ways if `$foo` is [not an integer](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq054). Therefore, there's not much point in quoting it properly other than for performance and to confine the arguments to a single word to reduce the likelihood of obscure side-effects possible in some shells.
If the input to any arithmetic context (including `((` or `let`), or `\[` test expression involving numeric comparisons can't be guaranteed then you must *always* [validate your input before evaluating the expression](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq054).
```
# POSIX
case $foo in
*[![:digit:]]*)
printf '$foo expanded to a non-digit: %s\n' "$foo" >&2
exit 1
;;
*)
[ $foo -gt 7 ]
esac
```
<<Anchor(pf8)>>
## grep foo bar | while read -r; do ((count++)); done
The code above looks OK at first glance, doesn't it? Sure, it's just a poor implementation of `grep -c`, but it's intended as a simplistic example. Changes to `count` won't propagate outside the `while` loop because each command in a pipeline is executed in a separate SubShell. This surprises almost every Bash beginner at some point.
POSIX doesn't specify whether or not the last element of a pipeline is evaluated in a subshell. Some shells such as ksh93 and Bash >= 4.2 with `shopt -s lastpipe` enabled will run the `while` loop in this example in the original shell process, allowing any side-effects within to take effect. Therefore, portable scripts must be written in such a way as to not depend upon either behavior.
For workarounds for this and similar issues, please see [Bash FAQ #24](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq024). It's a bit too long to fit here.
<<Anchor(pf9)>>
## if \[grep foo myfile\]
Many beginners have an incorrect intuition about `if` statements brought about by seeing the very common pattern of an `if` keyword followed immediately by a `\[` or `\[\[`. This convinces people that the `\[` is somehow part of the `if` statement's syntax, just like parentheses used in C's `if` statement.
This is *not* the case! `if` takes a *command*. `\[` is a command, not a syntax marker for the `if` statement. It's equivalent to the `test` command, except that the final argument must be a `\]`. For example:
```
# POSIX
if [ false ]; then echo "HELP"; fi
if test false; then echo "HELP"; fi
```
are equivalent -- both checking that the argument "false" is non-empty. In both cases HELP will always be printed, to the surprise of programmers from other languages guessing about shell syntax.
The syntax of an `if` statement is:
```
if COMMANDS
then
elif # optional
then
else # optional
fi # required
```
Once again, `\[` is a command. It takes arguments like any other regular *simple command*. `if` is a *compound command*which contains other commands -- and **there is no \[** in its syntax!
While bash has a builtin command `\[` and thus `knows` about `\[` it has nothing special to do with `\]`. Bash only passes `\]` as argument to the `\[` command, which requires `\]` to be the last argument only to make scripts look better.
There may be zero or more optional `elif` sections, and one optional `else` section.
The `if` compound command is made up of two or more sections containing *lists* of commands, each delimited by a `then`, `elif`, or `else` keyword, and is terminated by the `fi` keyword. The exit status of the final command of the first section and each subsequent `elif` section determines whether each corresponding `then` section is evaluated. Another `elif` is evaluated until one of the `then` sections is executed. If no `then` section is evaluated, then the `else` branch is taken, or if no `else` is given, the `if` block is complete and the overall `if` command returns 0 (true).
If you want to make a decision based on the output of a `grep` command, you do *not* want to enclose it in parentheses, brackets, backticks, or *any other* syntax! Just use `grep` as the `COMMANDS` after the `if`, like this:
```
if grep -q fooregex myfile; then
...
fi
```
If the `grep` matches a line from `myfile`, then the exit code will be 0 (true), and the `then` part will be executed. Otherwise, if there are no matches, `grep` will return non-zero and the overall `if` command will be zero.
**See also:**
`* BashGuide/TestsAndConditionals`
`* `[`http://wiki.bash-hackers.org/syntax/ccmd/if_clause`](http://wiki.bash-hackers.org/syntax/ccmd/if_clause)
<<Anchor(pf10)>>
== if \[bar="$foo"\]; then ... ==
```
[bar="$foo"] # Wrong!
[ bar="$foo" ] # Still wrong!
```
As explained in the previous example, `\[` is a command (which can be proven with `type -t \[` or `whence -v \[`). Just like with any other simple command, Bash expects the command to be followed by a space, then the first argument, then another space, etc. You can't just run things all together without putting the spaces in! Here is the correct way:
```
if [ bar = "$foo" ]; then ...
```
Each of `bar`, `=`, the expansion of `"$foo"`, and `\]` is a separate [argument](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/arguments)to the `\[` command. There must be whitespace between each pair of arguments, so the shell knows where each argument begins and ends.
<<Anchor(pf11)>>
== if \[ \[ a = b \] && \[ c = d \] \]; then ... == Here we go again. `\[` is a *command*. It is not a syntactic marker that sits between `if` and some sort of C-like "condition". Nor is it used for grouping. You cannot take C-like `if` commands and translate them into Bash commands just by replacing parentheses with square brackets!
If you want to express a compound conditional, do this:
```
if [ a = b ] && [ c = d ]; then ...
```
Note that here we have two *commands* after the `if`, joined by an `&&` (logical AND, shortcut evaluation) operator. It's precisely the same as:
```
if test a = b && test c = d; then ...
```
If the first `test` command returns false, the body of the `if` statement is not entered. If it returns true, then the second `test` command is run; and if that also one returns true, then the body of the `if` statement *will* be entered. (C programmers are already familiar with `&&`. Bash uses the same *short-circuit evaluation*. Likewise `||` does short-circuit evaluation for the *OR* operation.)
The \[\[BashFAQ/031|[keyword](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/keyword)*does* permit the use of `&&`, so it could also be written this way:
```
if [[ a = b && c = d ]]; then ...
```
See [pitfall #6](#pf6 "wikilink") for a pitfall related to *tests* combined with conditional operators.
<<Anchor(pf12)>>
## read $foo
You don't use a `$` before the variable name in a `read` command. If you want to put data into the variable named `foo`, you do it like this:
`. `
```
read foo
```
Or more safely:
`. `
```
IFS= read -r foo
```
`read $foo` would read a line of input and put it in the variable(s) whose name(s) are in `$foo`. This might be useful if you actually intended `foo` to be a [reference](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq006)to some other variable; but in the majority of cases, this is simply a bug.
<<Anchor(pf13)>>
## cat file | sed s/foo/bar/ > file
You **cannot** read from a file and write to it in the same pipeline. Depending on what your pipeline does, the file may be clobbered (to 0 bytes, or possibly to a number of bytes equal to the size of your operating system's pipeline buffer), or it may grow until it fills the available disk space, or reaches your operating system's file size limitation, or your quota, etc.
If you want to make a change to a file safely, other than appending to the end of it, use a text editor.
`. `
```
printf %s\\n ',s/foo/bar/g' w q | ed -s file
```
If you are doing something that cannot be done with a text editor there *must* be a temporary file created at some point(\*). For example, the following is completely portable:
`. `
```
sed 's/foo/bar/g' file > tmpfile && mv tmpfile file
```
The following will *only* work on GNU sed 4.x:
`. `
```
sed -i 's/foo/bar/g' file(s)
```
Note that this also creates a temporary file, and does the same sort of renaming trickery -- it just handles it transparently.
And the following equivalent command requires perl 5.x (which is probably more widely available than GNU sed 4.x):
`. `
```
perl -pi -e 's/foo/bar/g' file(s)
```
For more details on replacing contents of files, please see [Bash FAQ #21](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq021).
(\*) `sponge` from [moreutils](http://packages.debian.org/sid/moreutils "wikilink") uses this example in its manual:
`. `
```
sed '...' file | grep '...' | sponge file
```
Rather than using a temporary file plus an atomic `mv`, this version "soaks up" (the actual description in the manual!) all the data, before opening and writing to the `file`. This version will cause data loss if the program or system crashes during the write operation, because there's no copy of the original file on disk at that point.
Using a temporary file + `mv` still incurs a slight risk of data loss in case of a system crash / power loss; to be 100% certain that either the old or the new file will survive a power loss, you must use `sync` before the `mv`.
<<Anchor(pf14)>>
## echo $foo
This relatively innocent-looking command causes *massive*confusion. Because the `$foo` isn't [quoted](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes), it will not only be subject to WordSplitting, but also file [globbing](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob). This misleads Bash programmers into thinking their variables *contain* the wrong values, when in fact the variables are OK -- it's just the word splitting or filename expansion that's messing up their view of what's happening.
`. `
```
msg="Please enter a file name of the form *.zip"
echo $msg
```
This message is split into words and any globs are expanded, such as the \*.zip. What will your users think when they see this message:
`. `
```
Please enter a file name of the form freenfss.zip lw35nfss.zip
```
To demonstrate:
`. `
```
var="*.zip" # var contains an asterisk, a period, and the word "zip"
echo "$var" # writes *.zip
echo $var # writes the list of files which end with .zip
```
In fact, the `echo` command cannot be used with absolute safety here. If the variable contains `-n` for example, `echo` will consider that an option, rather than data to be printed. The only absolutely *sure* way to print the value of a variable is using `printf`:
`. `
```
printf "%s\n" "$foo"
```
<<Anchor(pf15)>>
== $foo=bar == No, you don't assign a variable by putting a `$` in front of the variable name. This isn't perl.
<<Anchor(pf16)>>
== foo = bar == No, you can't put spaces around the `=` when assigning to a variable. This isn't C. When you write `foo = bar` the shell splits it into three words. The first word, `foo`, is taken as the command name. The second and third become the arguments to that command.
Likewise, the following are also wrong:
`. `
```
foo= bar # WRONG!
foo =bar # WRONG!
$foo = bar; # COMPLETELY WRONG!
foo=bar # Right.
foo="bar" # More Right.
```
<<Anchor(pf17)>>
## echo <<EOF
A here document is a useful tool for embedding large blocks of textual data in a script. It causes a redirection of the lines of text in the script to the standard input of a command. Unfortunately, `echo` is not a command which reads from stdin.
`. `
```
# This is wrong:
echo <&2 "Can't read /net. Make sure you've logged in to the Samba network, and try again."; exit 1; }
do_stuff
more_stuff
```
Note there's a required space between `{` and `echo`, and a required `;` before the closing `}`. You could also write a `die` function, if you prefer.
Some people also like to enable [set -e](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq105) to make their scripts abort on *any* command that returns non-zero, but this can be [rather tricky to use correctly](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq105) (since many common commands may return a non-zero for a warning condition, which you may not want to treat as fatal).
By the way, if you're changing directories a lot in a Bash script, be sure to read the Bash help on `pushd`, `popd`, and `dirs`. Perhaps all that code you wrote to manage `cd`'s and `pwd`'s is completely unnecessary.
Speaking of which, compare this:
`. `
```
find ... -type d -print0 | while IFS= read -r -d '' subdir; do
here=$PWD
cd "$subdir" && whatever && ...
cd "$here"
done
```
With this:
`. `
```
find ... -type d -print0 | while IFS= read -r -d '' subdir; do
(cd "$subdir" || exit; whatever; ...)
done
```
Forcing a SubShell here causes the `cd` to occur only in the subshell; for the next iteration of the loop, we're back to our normal location, regardless of whether the `cd` succeeded or failed. We don't have to change back manually, and we aren't stuck in a neverending string of `... && ...` logic preventing the use of other conditionals. The subshell version is simpler and cleaner (albeit a tiny bit slower).
Another approach is to `cd` unconditionally to where we're supposed to be, at the start of each loop iteration:
`. `
```
here=$PWD
find ... -type d -print0 | while IFS= read -r -d '' subdir; do
cd "$here" || continue
cd "$subdir" || continue
whatever
...
done
```
At least this way, we can `continue` to the next loop iteration and don't have to string an indefinite series of `&&` together to ensure that we reach the `cd` at the end of the loop body.
<<Anchor(pf20)>>
## \[ bar
"$foo" \] == The `==` operator is not valid for the POSIX `\[` command. Use `=` or the \[\[BashFAQ/031|[keyword](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/keyword)instead.
`. `
```
[ bar = "$foo" ] && echo yes
[[ bar == $foo ]] && echo yes
```
In Bash, `\[ "$x" == y \]` is accepted as an extension, which often leads Bash programmers to think it's the correct syntax. It's not; it's a [Bashism](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashism). If you're going to use Bashisms, you might as well just use `\[\[` instead.
<<Anchor(pf21)>>
## for i in {1..10}; do ./something &; done
You *cannot* put a `;` immediately after an `&`. Just remove the extraneous `;` entirely.
`. `
```
for i in {1..10}; do ./something & done
```
Or:
`. `
```
for i in {1..10}; do
./something &
done
```
`&` already functions as a command terminator, just like `;` does. And you cannot mix the two.
In general, a `;` can be replaced by a newline, but not all newlines can be replaced by `;`.
<<Anchor(pf22)>>
## cmd1 && cmd2 || cmd3
Some people try to use `&&` and `||` as a shortcut syntax for `if ... then ... else ... fi`, perhaps because they think they are being clever. For instance,
`. `
```
# WRONG!
[[ -s $errorlog ]] && echo "Uh oh, there were some errors." || echo "Successful."
```
However, this construct is *not* completely equivalent to `if ... fi` in the general case. The command that comes after the `&&` also generates an exit status, and if that exit status isn't "true" (0), then the command that comes after the `||` will *also* be invoked. For example:
`. `
```
i=0
true && ((i++)) || ((i--)) # WRONG!
echo "$i" # Prints 0
```
What happened here? It looks like `i` should be 1, but it ends up 0. Why? Because both the `i++` *and* the `i--` were executed. The `((i++))` command has an exit status, and that exit status is derived from a C-like evaluation of the expression inside the parentheses. That expression's value happens to be 0 (the initial value of `i`), and in C, an expression with an integer value of 0 is considered *false*. So `((i++))` (when `i` is 0) has an exit status of 1 (false), and therefore the `((i--))` command is executed as well.
Another clever person thinks that we can fix it by using the pre-increment operator, since the exit status from `++i` (with `i` initially 0) is true:
`. `
```
i=0
true && (( ++i )) || (( --i )) # STILL WRONG!
echo "$i" # Prints 1 by dumb luck
```
But that's missing the point of the example. It just *happens*to work by *coincidence*, and you *cannot* rely on `x && y || z` if `y` has **any** chance of failure! (This example still fails if we initialize `i` to -1 instead of 0.)
If you need safety, or if you simply aren't sure how this works, or if *anything* in the preceding paragraphs wasn't completely clear, please just use the simple `if ... fi` syntax in your programs.
`. `
```
i=0
if true; then
((i++))
else
((i--))
fi
echo "$i" # Prints 1
```
This section also applies to Bourne shell, here is the code that illustrates it:
`. `
```
# WRONG!
true && { echo true; false; } || { echo false; true; }
```
Output is two lines "true" and "false", instead the single line "true".
<<Anchor(pf23)>>
## echo "Hello World!"
The problem here is that, in an interactive Bash shell (in versions prior to 4.3), you'll see an error like:
`. `
```
bash: !": event not found
```
This is because, in the default settings for an interactive shell, Bash performs csh-style history expansion using the exclamation point. This is **not** a problem in shell scripts; only in interactive shells.
Unfortunately, the obvious attempt to "fix" this won't work:
`. `
```
$ echo "hi\!"
hi\!
```
The easiest solution is unsetting the `histexpand` option: this can be done with `set +H` or `set +o histexpand`
`. Question: Why is playing with `histexpand` more apropriate than single quotes?`
` . `*`I personally ran into this issue when I was manipulating song files, using commands like`*
` `
```
mp3info -t "Don't Let It Show" ...
mp3info -t "Ah! Leah!" ...
```
` `*`Using single quotes is extremely inconvenient because of all the songs with apostrophes in their titles. Using double quotes ran into the history expansion issue. (And imagine a file that has both in its name. The quoting would be atrocious.) Since I never actually`*`use`*`history expansion, my personal preference was to turn it off in `~/.bashrc`.`*` -- GreyCat`
These solutions will work:
`. `
```
echo 'Hello World!'
```
or
`. `
```
set +H
echo "Hello World!"
```
or
`. `
```
histchars=
```
Many people simply choose to put `set +H` or `set +o histexpand` in their `~/.bashrc` to deactivate history expansion permanently. This is a personal preference, though, and you should choose whatever works best for you.
Another solution is:
`. `
```
exmark='!'
echo "Hello, world$exmark"
```
In Bash 4.3 and newer, a double quote following `!` no longer triggers history expansion, but history expansion is still performed within double quotes, so while `echo "Hello World!"` is OK, these will still be a problem:
`. `
```
echo "Hello, World!(and the rest of the Universe)"
echo "foo!'bar'"
```
<<Anchor(pf24)>>
## for arg in $\*
Bash (like all Bourne shells) has a special syntax for referring to the list of positional parameters one at a time, and `$\*` isn't it. Neither is `$@`. Both of those expand to the list of words in your script's parameters, not to each parameter as a separate word.
The correct syntax is:
`. `
```
for arg in "$@"
# Or simply:
for arg
```
Since looping over the positional parameters is such a common thing to do in scripts, `for arg` defaults to `for arg in "$@"`. The double-quoted `"$@"` is special magic that causes each parameter to be used as a single word (or a single loop iteration). It's what you should be using at least 99% of the time.
Here's an example:
`. `
```
# Incorrect version
for x in $*; do
echo "parameter: '$x'"
done
$ ./myscript 'arg 1' arg2 arg3
parameter: 'arg'
parameter: '1'
parameter: 'arg2'
parameter: 'arg3'
```
It should have been written:
`. `
```
# Correct version
for x in "$@"; do
echo "parameter: '$x'"
done
# or better:
for x do
echo "parameter: '$x'"
done
$ ./myscript 'arg 1' arg2 arg3
parameter: 'arg 1'
parameter: 'arg2'
parameter: 'arg3'
```
<<Anchor(pf25)>>
## function foo()
This works in some shells, but not in others. You should *never* combine the keyword `function` with the parentheses `()` when defining a function.
Bash (at least some versions) will allow you to mix the two. Most of the shells won't accept that (zsh 4.x and perhaps above will - for example). Some shells will accept `function foo`, but for maximum portability, you should always use:
`. `
```
foo() {
...
}
```
<<Anchor(pf26)>>
## echo "~"
Tilde expansion only applies when '~' is unquoted. In this example echo writes '~' to stdout, rather than the path of the user's home directory.
Quoting path parameters that are expressed relative to a user's home directory should be done using $HOME rather than '~'. For instance consider the situation where $HOME is "/home/my photos".
`. `
```
"~/dir with spaces" # expands to "~/dir with spaces"
~"/dir with spaces" # expands to "~/dir with spaces"
~/"dir with spaces" # expands to "/home/my photos/dir with spaces"
"$HOME/dir with spaces" # expands to "/home/my photos/dir with spaces"
```
<<Anchor(pf27)>>
== local varname=$(command) == When declaring a local variable in a function, the `local` acts as a command in its own right. This can sometimes interact oddly with the rest of the line -- for example, if you wanted to capture the exit status (`$?`) of the CommandSubstitution, you can't do it. `local`'s exit status masks it.
It's best to use separate commands for this:
`. `
```
local varname
varname=$(command)
rc=$?
```
This is also true of `export`, which will similarly mask the exit status.
The next pitfall describes another issue with this syntax:
<<Anchor(pf28)>>
== export foo=~/bar == [Tilde expansion](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/tildeexpansion) (with or without a username) is only guaranteed to occur when the tilde appears at the beginning of a [word](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/arguments), either by itself or followed by a slash. It is also guaranteed to occur when the tilde appears immediately after the `=` in an assignment.
However, the `export` and `local` commands *do not constitute an assignment*. So, in some shells (like Bash), `export foo=~/bar` will undergo tilde expansion; in others (like dash), it will not.
`. `
```
foo=~/bar; export foo # Right!
export foo="$HOME/bar" # Right!
```
<<Anchor(pf29)>>
## sed 's/$foo/good bye/'
In [single quotes](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes), bash parameter expansions like `$foo` do not get expanded. That is the purpose of single quotes, to protect characters like `$` from the shell.
Change the quotes to double quotes:
`. `
```
foo="hello"; sed "s/$foo/good bye/"
```
But keep in mind, if you use double quotes you might need to use more escapes. See the [Quotes](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes)page.
<<Anchor(pf30)>>
## tr \[A-Z\] \[a-z\]
There are (at least) three things wrong here. The first problem is that `\[A-Z\]` and `\[a-z\]` are seen as [glob](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob)s by the shell. If you don't have any single-lettered filenames in your current directory, it'll seem like the command is correct; but if you do, things will go wrong. Probably at 0300 hours on a weekend.
The second problem is that this is not really the correct notation for `tr`. What this actually does is translate '\[' into '\['; anything in the range A-Z into a-z; and '\]' into '\]'. So you don't even need those brackets, and the first problem goes away.
The third problem is that depending on the [locale](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/locale), A-Z or a-z may not give you the 26 ASCII characters you were expecting. In fact, in some locales z is in the middle of the alphabet! The solution to this depends on what you want to happen:
`. `
```
# Use this if you want to change the case of the 26 latin letters
LC_COLLATE=C tr A-Z a-z
# Use this if you want the case conversion to depend upon the locale, which might be more like what a user is expecting
tr '[:upper:]' '[:lower:]'
```
The quotes are required on the second command, to avoid [globbing](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob).
<<Anchor(pf31)>>
## ps ax | grep gedit
The fundamental problem here is that the name of a running process is inherently unreliable. There could be more than one legitimate gedit process. There could be something else disguising itself as gedit (changing the reported name of an executed command is trivial). For *real* answers to this, see ProcessManagement.
The following is the quick and dirty stuff.
Searching for the PID of (for example) gedit, many people start with
```
$ ps ax | grep gedit
10530 ? S 6:23 gedit
32118 pts/0 R+ 0:00 grep gedit
```
which, depending on a RaceCondition, often yields grep itself as a result. To filter grep out:
```
ps ax | grep -v grep | grep gedit # will work, but ugly
```
An alternative to this is to use:
```
ps ax | grep '[g]edit' # quote to avoid shell GLOB
```
This will ignore the grep itself in the process table as that is \[g\]edit and grep is looking for gedit once evaluated.
On GNU/Linux, the parameter -C can be used instead to filter by commandname:
```
$ ps -C gedit
PID TTY TIME CMD
10530 ? 00:06:23 gedit
```
But why bother when you could just use pgrep instead?
```
$ pgrep gedit
10530
```
Now in a second step the PID is often extracted by awk or cut:
```
$ ps -C gedit | awk '{print $1}' | tail -n1
```
but even that can be handled by some of the trillions of parameters for ps:
```
$ ps -C gedit -opid=
10530
```
If you're stuck in 1992 and aren't using pgrep, you could use the ancient, obsolete, deprecated pidof (GNU/Linux only) instead:
```
$ pidof gedit
10530
```
and if you need the PID to kill the process, *pkill* might be interesting for you. Note however that, for example, `pgrep/pkill ssh` would also find processes named sshd, and you wouldn't want to kill those.
Unfortunately some programs aren't started with their name, for example firefox is often started as firefox-bin, which you would need to find out with - well - **ps ax | grep firefox**. :) Or, you can stick with pgrep by adding some parameters:
```
$ pgrep -fl firefox
3128 /usr/lib/firefox/firefox
7120 /usr/lib/firefox/plugin-container /usr/lib/flashplugin-installer/libflashplayer.so -greomni /usr/lib/firefox/omni.ja 3128 true plugin
```
Please read ProcessManagement. Seriously.
<<Anchor(pf32)>>
## printf "$foo"
This isn't wrong because of [quotes](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes), but because of a *format string exploit*. If `$foo` is not strictly under your control, then any `\\` or `%` characters in the variable may cause undesired behavior.
Always supply your own format string:
```
printf %s "$foo"
printf '%s\n' "$foo"
```
<<Anchor(pf33)>>
## for i in {1..$n}
The BashParser performs BraceExpansion *before* any other expansions or substitutions. So the brace expansion code sees the literal `$n`, which is not numeric, and therefore it doesn't expand the curly braces into a list of numbers. This makes it nearly impossible to use brace expansion to create lists whose size is only known at run-time.
Do this instead:
```
for ((i=1; i<=n; i++)); do
...
done
```
In the case of simple iteration over integers, an arithmetic `for` loop should almost always be preferred over brace expansion to begin with, because brace expansion pre-expands every argument which can be slower and unnecessarily consumes memory.
<<Anchor(pf34)>>
## if [$foo = $bar](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/foo-bar) (depending on intent)
When the right-hand side of an `=` operator inside \[\[BashFAQ/031|[](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/)is not quoted, bash does [pattern matching](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob) against it, instead of treating it as a string. So, in the code above, if `bar` contains `\*`, the result will *always* be true. If you want to check for equality of strings, the right-hand side should be quoted:
```
if [[ $foo = "$bar" ]]
```
If you want to do pattern matching, it might be wise to choose variable names that indicate the right-hand side contains a pattern. Or use comments.
It's also worth pointing out that if you quote the right-hand side of `=~` it *also* forces a simple string comparison, rather than a regular expression matching. This leads us to:
<<Anchor(pf35)>>
## if [$foo =~ 'some RE'](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/foo-some-re)
The quotes around the right-hand side of the `=~` operator cause it to become a string, rather than a RegularExpression. If you want to use a long or complicated regular expression and avoid lots of backslash escaping, put it in a variable:
```
re='some RE'
if [[ $foo =~ $re ]]
```
This also works around the difference in how `=~` works across different versions of bash. Using a variable avoids some nasty and subtle problems.
The same problem occurs with [pattern matching](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob) inside `[`: <pre> \[\[ $foo = "\*.glob"](`:__[[_$foo_=_"*.glob" "wikilink") # Wrong! \*.glob is treated as a literal string. [$foo = \*.glob]($foo_=_*.glob "wikilink") # Correct. \*.glob is treated as a glob-style pattern.
<<Anchor(pf36)>>
## \[ -n $foo \] or \[ -z $foo \]
When using the `\[` command, you **must** [quote](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/quotes)each substitution that you give it. Otherwise, `$foo` could expand to 0 words, or 42 words, or any number of words that isn't 1, which breaks the syntax.
```
[ -n "$foo" ]
[ -z "$foo" ]
[ -n "$(some command with a "$file" in it)" ]
# [[ doesn't perform word-splitting or glob expansion, so you could also use:
[[ -n $foo ]]
[[ -z $foo ]]
```
<<Anchor(pf37)>>
## [-e "$broken\_symlink"](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/-e-broken-symlink) returns 1 even though $broken\_symlink exists
Test follows symlinks, therefore if a symlink is broken, i.e. it points to a file that doesn't exists or is in a directory you don't have access to, test -e returns 1 for it even though it exists.
In order to work around it (and prepare against it) you should use:
```
# bash/ksh/zsh
[[ -e "$broken_symlink" || -L "$broken_symlink" ]]
# POSIX sh+test
[ -e "$broken_symlink" ] || [ -L "$broken_symlink" ]
```
<<Anchor(pf38)>>
## ed file <<<"g/d\\{0,3\\}/s//e/g" fails
The problem caused because ed doesn't accept 0 for \\{0,3\\}.
You can check that the following do work:
```
ed file <<<"g/d\{1,3\}/s//e/g"
```
Note that this happens even though POSIX states that BRE (which is the Regular Expression flavor used by ed) [should accept 0 as the minimum number of occurrences (see section 5)](http://www.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_06 "wikilink").
<<Anchor(pf39)>>
## expr sub-string fails for "match"
This works reasonably well
---
most of the time
```
word=abcde
expr "$word" : ".\(.*\)"
bcde
```
But WILL fail for the word "match"
```
word=match
expr "$word" : ".\(.*\)"
```
The problem is "match" is a keyword. Solution (GNU only) is prefix with a '+'
```
word=match
expr + "$word" : ".\(.*\)"
atch
```
Or, y'know, stop using `expr`. You can do everything `expr` does by using [Parameter Expansion](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq073). What's that thing up there trying to do? Remove the first letter of a word? That can be done in POSIX shells using PE or Substring Expansion:
```
$ word=match
$ echo "${word#?}" # PE
atch
$ echo "${word:1}" # SE
atch
```
Seriously, there's no excuse for using `expr` unless you're on Solaris with its non-POSIX-conforming `/bin/sh`. It's an external process, so it's much slower than in-process string manipulation. And since nobody uses it, nobody understands what it's doing, so your code is obfuscated and hard to maintain.
<<Anchor(pf40)>>
## On UTF-8 and Byte-Order Marks (BOM)
**In general:** Unix UTF-8 text does not use BOM. The encoding of plain text is determined by the locale or by mime types or other metadata. While the presence of a BOM would not normally damage a UTF-8 document meant only for reading by humans, it is problematic (often syntactically illegal) in any text file meant to be interpreted by automated processes such as scripts, source code, configuration files, and so on. Files starting with BOM should be considered equally foreign as those with MS-DOS linebreaks.
**In shell scripting:** 'Where UTF-8 is used transparently in 8-bit environments, the use of a BOM will interfere with any protocol or file format that expects specific ASCII characters at the beginning, such as the use of "#!" of at the beginning of Unix shell scripts.' [http://unicode.org/faq/utf\_bom.html#bom5](http://unicode.org/faq/utf_bom.html#bom5)
<<Anchor(pf41)>>
== content=$(<file) == There isn't anything wrong with this expression, but you should be aware that command substitutions (all forms:
```
`...`
```
, `$(...)`, `$(<file)`,
```
`>logfile 2>&1
```
See [a more in-depth explanation](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq055), [Copy descriptor explained](http://wiki.bash-hackers.org/scripting/copydescriptor "wikilink"), and [BashGuide - redirection](BashGuide/InputAndOutput#Redirection "wikilink").
<<Anchor(pf44)>>
## cmd; (( ! $? )) || die
`$?` is only required if you need to retrieve the exact status of the previous command. If you only need to test for success or failure (any non-zero status), just test the command directly. e.g.:
```
if cmd; then
...
fi
```
Checking an exit status against a list of alternatives might follow a pattern like this:
```
cmd
status=$?
case $status in
0)
echo success >&2
;;
1)
echo 'Must supply a parameter, exiting.' >&2
exit 1
;;
*)
echo "Unknown error $status, exiting." >&2
exit "$status"
esac
```
<<Anchor(pf45)>>
== y=$(( array\[$x\] )) == Due to [the POSIX wording](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04 "wikilink") of arithmetic expansion (which calls for expansion of command substitutions *after* parameter expansion), expansion of an array subscript inside an arithmetic expansion can lead to code injection exploits.
Yeah, that's a lot of big, confusing words. Here's how it breaks:
```
$ x='$(date >&2)' # redirection is just so we can see everything happen
$ y=$((array[$x])) # array doesn't even have to exist
Mon Jun 2 10:49:08 EDT 2014
```
Quoting `"$x"` won't help, either:
```
$ y=$((array["$x"]))
Mon Jun 2 10:51:03 EDT 2014
```
The two tricks that *do* work are:
```
# 1. Escape the $x so it isn't expanded prematurely.
$ y=$((array[\$x]))
# 2. Use the full ${array[$x]} syntax.
$ y=$((${array[$x]}))
```
<<Anchor(pf46)>>
## read num; echo $((num+1))
Always validate your input (see [BashFAQ/054](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/bashfaq054)) before using num in an arithmetic context as it allows code injection.
```
$ echo 'a[$(echo injection >&2)]' | bash -c 'read num; echo $((num+1))'
injection
1
```
<<Anchor(pf47)>>
== IFS=, read -ra fields <<< "$csv\_line" == Unbelievable as it may seem, POSIX requires the treatment of [IFS](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/ifs)as a field *terminator*, rather than a field *separator*. What this means in our example is that if there's an empty field at the end of the input line, it will be discarded:
```
$ IFS=, read -ra fields <<< "a,b,"
$ declare -p fields
declare -a fields='([0]="a" [1]="b")'
```
Where did the empty field go? It was eaten for historical reasons ("because it's always been that way"). This behavior is not unique to bash; all conformant shells do it. A non-empty field is properly scanned:
```
$ IFS=, read -ra fields <<< "a,b,c"
$ declare -p fields
declare -a fields='([0]="a" [1]="b" [2]="c")'
```
So, how do we work around this nonsense? As it turns out, appending an IFS character to the end of the input string will force the scanning to work. If there was a trailing empty field, the extra IFS character "terminates" it so that it gets scanned. If there was a trailing non-empty field, the IFS character creates a new, empty field that gets dropped.
```
$ input="a,b,"
$ IFS=, read -ra fields <<< "$input,"
$ declare -p fields
declare -a fields='([0]="a" [1]="b" [2]="")'
```
<<Anchor(pf48)>>
== export CDPATH=.:~/myProject == Do not export CDPATH.
Setting CDPATH in .bashrc is not an issue, but exporting it will cause any bash or sh script you run, that happen to use `cd`, to potentially change behaviour.
There are two problems. A script that does the following:
```
cd some/dir || exit
cmd to be run in some/dir
```
may change directory to `~/myProject/some/dir` instead of `./some/dir`, depending on what directories exist at the time. So the `cd` may succeed and take the script to the wrong directory, with potentially harmful effects of the following commands which now run in a different directory than intended.
The second problem is when `cd` is run in a context where the output is captured:
```
output=$(cd some/dir && some command)
```
As a side-effect when CDPATH is set, `cd` will output something like `/home/user/some/dir` to stdout to indicate that it found a directory through CDPATH, which in turn will end up in the output variable along with the intended output of `some command`.
A script can make itself immune to a CDPATH inherited from the environment by always prepending `./` to relative paths, or run `unset CDPATH` at the start of the script, but don't assume every scripter has taken this pitfall into account, so don't export CDPATH.
<<Anchor(pf49)>>
== OIFS="$IFS"; ...; IFS="$OIFS" == Directly assigning a variable's value to a temporary variable isn't alone enough to restore its state. The assignment will always result in a *set* but *empty*temporary variable even if the initial variable was unset. This is a particular problem for IFS because an *empty* IFS has a completely different meaning from an *unset* IFS, and setting IFS to a temporary value for a command or two is a common requirement.
An easy workaround is to designate a prefix to distinguish set from unset vars, then strip it when finished.
```
# oIFS is unset or null implies IFS is null.
typeset oIFS=${IFS+_${IFS}}
IFS=/; echo "${array[*]}"
${oIFS:+'false'} unset -v IFS || IFS=${oIFS#_}
```
A local variable is usually preferable when possible.
```
f() {
local IFS
IFS=/; echo "${array[*]}"
}
f
```
Subshells are another possibility.
```
( IFS=/; echo "${array[*]}" )
```
<<Anchor(pf50)>>
== hosts=( $(aws ...) ) == It is not safe to populate an array with a raw `$(...)` CommandSubstitution. The output of the command undergoes word splitting (on *all* whitespace, even ones that are inside quotes) and then [globbing](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob). If there's a word like `\*` or `eh?` or `\[abc\]` in the result, it will be expanded based on filenames in the current working directory.
To select a replacement, you need to know whether the command writes its output on a single line, or multiple lines. If it's a single line:
```
read -ra hosts < <(aws ...)
```
If it's multiple lines (and you're targeting bash 4.0 or later):
```
readarray -t hosts < <(aws ...)
```
If it's multiple lines (and you want compatibility with bash 3.x, *or* want your command's exit status to be reflected in success or failure of the `read` operation without depending on behavior only available in bash 4.4 and newer):
```
IFS=$'\n' read -r -d '' -a hosts < <(aws ... && printf '\0')
```
This will prevent globbing. It still won't help you if you needed to avoid splitting on quoted whitespace, but unfortunately *nothing*bash can do handles that case. For generalized CSV (comma-separated value) file handling, you really need to switch to a language that has a dedicated CSV input library.
<<Anchor(pf51)>>
## Non-atomic writes with xargs -P
GNU `xargs` supports running multiple jobs in parallel. `-P n` where **n** is the number of jobs to run in parallel.
```
seq 100 | xargs -n1 -P10 echo "$a" | grep 5
seq 100 | xargs -n1 -P10 echo "$a" > myoutput.txt
```
This will work fine for many situations but has a deceptive flaw: If `$a` contains more than ~1000 characters, the `echo` may not be atomic (it may be split into multiple `write()` calls), and there is a risk that two lines will be mixed.
```
$ perl -e 'print "a"x2000, "\n"' > foo
$ strace -e write bash -c 'read -r foo < foo; echo "$foo"' >/dev/null
write(1, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 1008) = 1008
write(1, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 993) = 993
+++ exited with 0 +++
```
Obviously the same issue arises if there are multiple calls to `echo` or `printf`:
```
slowprint() {
printf 'Start-%s ' "$1"
sleep "$1"
printf '%s-End\n' "$1"
}
export -f slowprint
seq 10 | xargs -n1 -I {} -P4 bash -c "slowprint {}"
# Compare to no parallelization
seq 10 | xargs -n1 -I {} bash -c "slowprint {}"
# Be sure to see the warnings in the next Pitfall!
```
Outputs from the parallel jobs are mixed together, because each job consists of two (or more) separate `write()` calls.
If you need the outputs unmixed, it is therefore recommended to use a tool that guarantees output will be serialized (such as GNU Parallel).
For further details see [a demonstration of the mixing problem](https://gist.github.com/ole-tange/88ae153797748b3618e2433377e2870a "wikilink").
<<Anchor(pf52)>>
## find . -exec sh -c 'echo {}' \\;
This command contains a CodeInjection vulnerability. The filename that is found by `find` is injected into a shell command and parsed by `sh`. If the filename contains shell metacharacters like `;` or `$( ... )` then the filename may be *executed as code* by `sh'.
The "slowprint" example in the previous Pitfall would have been a CodeInjection bug if the input weren't guaranteed to be integers.
To be more precise, [POSIX find](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html#tag_20_47 "wikilink") does not specify whether an argument which contains *more than* just `{}` is expanded. GNU `find` allows this CodeInjection to occur. Other implementations choose a safer path:
```
# uname -a
HP-UX imadev B.10.20 A 9000/785 2008897791 two-user license
# find /dev/null -exec sh -c 'echo {}' \;
{}
```
The correct approach is to *separate* the filename argument from the script argument:
```
find . -exec sh -c 'echo "$1"' x {} \;
```
<<Anchor(pf53)>>
## sudo mycmd > /myfile
[Redirection](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/redirection)is done *before* the command is executed. Usually that doesn't matter, but with `sudo` we have a command being executed as a different user than the redirection.
If the redirection must be executed with `sudo`-granted privileges, then you need a wrapper:
```
sudo sh -c 'mycmd > /myfile'
```
<<Anchor(pf54)>>
## sudo ls /foo/\*
This is very similar to the previous pitfall. [Globbing](https://www.reeltoreel.nl/knowledgebase/books/mediawiki-import/page/glob)is also done *before* the command is executed. If the directory isn't readable by your normal user privileges, then you may need the globbing to be done in a shell that has the `sudo`-granted privileges:
```
sudo sh -c 'ls /foo/*'
```
<<Anchor(pf55)>>
## myprogram 2>&-
**Do not** close stdin, stdout or stderr as a "shorthand" for redirecting to `/dev/null`. Write it out correctly.
```
myprogram 2>/dev/null
```
Why? Consider what happens when your program tries to write an error message to stderr. If stderr has been redirected to `/dev/null`, the write succeeds, and your program is free to carry on, secure in the knowledge that it has diligently reported the error condition.
But if stderr has been *closed*, then the write will fail. At that point, your program may do something unpredictable. It may carry on and ignore the failure, or it may immediately exit, considering the execution environment so broken that it cannot safely continue. Or whatever else the programmer decided the program should do when its world has become a dystopian hell.
All programs are assured that stdin, stdout and stderr will *exist* and will be readable/writable in an appropriate and reasonable manner. By closing one of them, you have violated your promise to this program. This is not acceptable.
Of course, an even better solution would be to actually log the errors somewhere, so you can go back and read them and figure out what's wrong.
<<Anchor(pf56)>>
## Using xargs without -0
`xargs` splits on whitespace. This is unfortunate because whitespace is allowed in filenames and commonly used by GUI users. `xargs` also treats `'` and `"` specially, which can also lead to problems:
```
touch Dad\'s\ \"famous\"\ 1\'\ pizza.txt
touch Dad\'s\ 12\"\ records.txt
touch 2\"x1\'\ wood.txt
touch 2\"x4\"\ wood.txt
```
Here `xargs` warns:
```
# Do not do this
$ find . -type f | xargs wc
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
```
Here `xargs` does not warn at all:
```
# Do not do this
echo * | xargs wc
find *famous* -type f | xargs wc
find *4* -type f | xargs wc
```
Instead use `xargs -0`:
```
# Do this instead
printf '%s\0' * | xargs -0 wc
find . -type f -name '*famous*' -print0 | xargs -0 wc
find . -type f -name '*4*' -exec wc {} +
```
If using `-0` is not simple, an alternative is to use GNU Parallel, which splits on \\n. And while \\n is also allowed in filenames they never occur unless your users are malicious. In any case: **If**you use `xargs` without `-0` put a comment in your code explaining why that is safe in your particular situation.
---
CategoryShell CategoryBashguide
# File permissions
## File permissions
One of the important things you can do to help secure your MediaWiki install, is ensure that the user you are running php as (often www-data if using debian) and the user you are running mysql as, does not have write access to any web accessible directory with php enabled.
- On unix-like systems, you can do this by ensuring that the mediawiki directory/files are owned by someone other than your web server user (www-data) or mysql server user. Depending on how you installed MediaWiki this may already be the case, but if not can be accomplished by doing `chown -R `` /path/to/MediaWiki/`where username is a user other than the webserver or mysql user (commonly you would use your own username provided mysql</> and php are not running as your username).
- **After doing that step, you may however need to change the owner of the image directory back to the php user,** as uploaded files need to go there, so MediaWiki needs to be able to write there (e.g. `chown -R www-data /path/to/MediaWiki/images`).
- Next you run `chmod -R go-w /path/to/MediaWiki` to remove write access from all other users besides the file owners.
- After doing that step you may need to re-enable write access to the images directory.
- Directories that MediaWiki needs write access to (such as $wgCacheDirectory if that feature is enabled) should be located outside of the web root. The exception being the images directory, which must be in the web root.
- However, it is important to disable php in the images directory. The details on how to do this varies with webserver, but on apache it can sometimes be accomplished by using `php_flag engine off` in a .htaccess</> file. If you do accomplish this via a config file in the images directory itself, you should ensure the config file is not writable by the webserver. See the section below on upload security for more details.
- Your LocalSettings.php file must be readable by the php user, however it should not be world readable, to prevent other processes from discovering your database password and other sensitive information. Like all MediaWiki files, the php user should not be able to write to LocalSettings.php.
## permissions
New install of MediaWiki 1.28 on Ubuntu 16.04. Permissions for /var/www/html set to 755, owned by me user:www-data.
Did a Download from Git install, before and after doing composer install, the file and directory permissions are readable and writable but not executable for group. Should I set them to 755 manually?
In your MediaWiki directory, the following SSH commands should work:
`find . -type f -exec chmod 644 {} \;`
`find . -type d -exec chmod 755 {} \;`
## securing
The following picks up on a fresh working LAMP installation under Red Hat 7 or clone (CentOS 7, Scientific Linux 7, Orcale 7, etc). Set Selinux to permissive for the installation.
` setenforce 0`
First get the Mediawiki version you want from [https://releases.wikimedia.org/mediawiki/](https://releases.wikimedia.org/mediawiki/), at time of writing latest is [https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.0.tar.gz](https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.0.tar.gz)and unpack it in /var/www/html/w.
Navigate to [https://www.example.com/w](https://www.example.com/w) and follow on-screen instructions to generate content used for LocalSettings.php. Create LocalSettings.php with
` vi /var/www/html/w/LocalSettings.php`
and paste content into file (i -> enter insert mode, CTRL+SHIFT+v to paste content, ESC -> to exit insert mode, ZZ(twice letter Z) to save and exit vi). Now secure LocalSettings.php with
` chown root:apache /var/www/html/w/LocalSettings.php`
` chmod 640 /var/www/html/w/LocalSettings.php`
Delete mw-config if it exists, since it is only used for first time setup of mediawiki.
` rm -rf /var/www/html/w/mw-config`
Enable use of .htaccess files by creating custom configuration file for Apache httpd.
` cat >> /etc/httpd/conf.d/custom.conf << EOF`
` `
` AllowOverride All`
` `
` EOF`
Now one should customize LocalSettings.php to one's taste. Here an overview of variables that can be customized: [https://www.mediawiki.org/wiki/Manual:Configuration\_settings](https://www.mediawiki.org/wiki/Manual:Configuration_settings)
Since we want to access our Mediawiki installation under [https://www.example.com/wiki](https://www.example.com/wiki) we need to set $wgArticlePath in LocalSettings.php. Just add the following line a the bottom of LocalSettings.php
` $wgArticlePath = "/wiki/$1";`
and update /etc/httpd/conf.d/ssl.conf by adding one line.
` `
` Alias /wiki /var/www/html/w/index.php # <-- only add this line`
### selinux
Now finish securing the Mediawiki installation. This \*may\* be needed for Selinux, e.g. database on different server, etc.
` setsebool -P httpd_can_network_connect 1`
` setsebool -P httpd_can_network_connect_db 1`
This \*is\* needed for Selinux to run Mediawiki
` setsebool -P httpd_builtin_scripting 1`
` setsebool -P httpd_execmem 1`
Set userrights and special Selinux rights, so Apache httpd has read access, but other users beside root don't.
` chown -R root:apache /var/www/html/`
` find /var/www/html/w -type d -exec chmod 750 {} \;`
` find /var/www/html/w -type f -exec chmod 640 {} \;`
Mediawiki writes to images and cache, so they need special write premissions.
` chown -R apache:apache /var/www/html/w/images`
` chown -R apache:apache /var/www/html/w/cache`
` semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/w/cache(/.*)?"`
` semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/w/images(/.*)?"`
` restorecon -R /var/www/html/w`
Found Selinux complaining about hugetlbfs, so add an exception.
` cd /var/log/audit`
` grep hugetlbfs audit.log | audit2allow -M hugetlbfs`
` semodule -i hugetlbfs.pp`
Now restart Apache httpd and set Selinux back to enforcing.
` setenforce 1`
` systemctl restart httpd`
Understandably this covers only the basics and Mediawiki offers thousands of ways to customize it further to one's taste and security needs.
Don't forget to make regular backups.
Further suggestions can be found here [https://www.pozzo-balbi.com/help/Mediawiki](https://www.pozzo-balbi.com/help/Mediawiki).
# Disable cpu core for light work
`# echo 0 | sudo tee /sys/bus/cpu/devices/cpu$COREID/online # Disable a CPU core to save battery and lower thermals. Works greats if your laptop is noisy and you're only doing light work`
# Rescan SCSI bus
`# echo "0 0 0" > /sys/class/scsi_host/host5/scan`
5 is changeable; it is the connection
# How to send email from command line in fixed font
To send email that is in fixed font you can use:
```
# (echo "Subject: bla"; echo "MIME-Version: 1.0"; echo "Content-Type: text/html"; echo "Content-Disposition: inline"; echo 'The code element') | sendmail joop@hotmail.com
```
Mind the /pre here.