Homelab

Here is how to reinstall GRUB after you have had to install windows, and windows has overwritten your master boot record (MBR) so that you can't get back into Linux.

boot from install CD (knoppix did not work in my case, use openSUSE CD)

sux - <-this is optional, just use 'su'
grub
find /boot/grub/stage1 // Will print something like (hd0,1)
or
find /boot/grub/menu.lst
root (hd0,1)
setup (hd0) <- ignore prompt for partition
quit

Boot & Disk

Enable harddisk spin-down / sleep mode / standby

Aan elke harddisk in een Linux systeem kun je een aparte time-out geven wanneer deze een spin-down moet gaan doen (standby / sleepmode).

Zodra een harddisk in standmode staat gebruikt deze fors minder stroom. Hiervoor kun je het volgende commando gebruiken

hdparm -S 60 /dev/sdd

Met bovenstaande commando wordt er een time-out toegekend van 5 minuten aan de systeem disk.

Zodra de systeem disk langer dan 5 minuten niet gebruikt wordt dan zal deze een spin-down gaan doen.

1 Minuut is 12 'eenheden' van 5 seconden.

bron: http://www.olino.org/articles/2007/02/25/het-bouwen-van-een-zuinige-server-deel-3

Boot & Disk

Linux Software RAID: growing filesystems and adding disks

Adding partitions

When new disks are added, existing raid partitions can be grown to use the new disks. After the new disk was partitioned, the RAID level 1/4/5/6 array can be grown for example using this command (assuming that before growing it contains three drives):

mdadm --add /dev/md1 /dev/sdb3
mdadm --grow --raid-devices=4 /dev/md1

The process can take even 10 hours. There is a critical section at start, which cannot be backed up. To allow recovery after unexpected power failure, an additional option --backup-file= can be specified.

Expanding existing partitions

It is possible to migrate the whole array to larger drives (e.g. 250 GB to 1 TB) by replacing one by one. In the end the number of devices will be the same, the data will remain intact, and you will have more space available to you.

Extending an existing RAID array

In order to increase the usable size of the array, you must increase the size of all disks in that array. Depending on the size of your disks, this may take days to complete. It is also important to note that while the array undergoes the resync process, it is vulnerable to irrecoverable failure if another drive were to fail. It would (of course) be a wise idea to completely back up your data before continuing.

First, choose a drive and completely remove it from the array

mdadm -f /dev/md0 /dev/sdd1
mdadm -r /dev/md0 /dev/sdd1

Next, partition the new drive so that you are using the amount of space you will eventually use on all new disks. For example, if you are going from 100 GB drives to 250 GB drives, you will want to partition the new 250 GB drive to use 250 GB, not 100 GB. Also, remember to set the partition type to 0xDA - Non-fs data (or 0xFD, Linux raid autodetect if you are still using the deprecated autodetect).

fdisk /dev/sde

Now add the new disk to the array:

mdadm --add /dev/md0 /dev/sde1

Allow the resync to fully complete before continuing. You will now have to repeat the above steps for *each* disk in your array. Once all of the drives in your array have been replaced with larger drives, we can grow the space on the array by issuing:

mdadm --grow /dev/md0 --size=max

The array now represents one disk using all of the new available space.

Extending the filesystem

Now that you have expanded the underlying partition, you must now resize your filesystem to take advantage of it. For an ext2/ext3 filesystem:

resize2fs /dev/md0

For a reiserfs filesystem:

resize_reiserfs /dev/md0

Please see filesystem documentation for other filesystems.

LVM: Growing the PV

LVM (logical volume manager) abstracts a logical volume (that a filesystem sits on) from the physical disk. If you are used to LVM then you are likely used to growing LVs (logical volumes), but what we grow here is the PV (physical volume) that sits on the md device (RAID array).

For further LVM documentation, please see the Linux LVM HOWTO

Growing the physical volume is trivial:

pvresize /dev/md0

A before-and-after example is:

root@barcelona:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               server1_vg
  PV Size               931.01 GB / not usable 558.43 GB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              95379
  Free PE               42849
  Allocated PE          52530
  PV UUID               BV0mGK-FRtQ-KTLv-aW3I-TllW-Pkiz-3yVPd1

root@barcelona:~# pvresize /dev/md0
Physical volume "/dev/md0" changed
1 physical volume(s) resized / 0 physical volume(s) not resized

root@barcelona:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               server1_vg
  PV Size               931.01 GB / not usable 1.19 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              238337
  Free PE               185807
  Allocated PE          52530
  PV UUID               BV0mGK-FRtQ-KTLv-aW3I-TllW-Pkiz-3yVPd1

The above is the PV part after md0 was grown from ~400GB to ~930GB (a 400GB disk to a 1TB disk). Note the PV Size descriptions before and after.

Once the PV has been grown (and hence the size of the VG, volume group, will have increased), you can increase the size of an LV (logical volume), and then finally the filesystem, eg:

lvextend -L +50G -n home_lv server1_vg
resize2fs /dev/server1_vg/home_lv

The above grows the home_lv logical volume in the server1_vg volume group by 50GB. It then grows the ext2/ext3 filesystem on that LV to the full size of the LV, as per Extending the filesystem above.

Boot & Disk

Partitions

Per 1 augustus 2010

/dev/sda (SDD-80GB)

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048     4208639     2103296   82  Linux swap / Solaris             swap
/dev/sda2   *     4208640    46153727    20972544   83  Linux                            /
/dev/sda3        46153728   156301311    55073792   83  Linux                            /home

/dev/sdb (HDD-250GB)

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1              63    62637434    31318686    7  HPFS/NTFS/exFAT
/dev/sdb2       467419136   469506047     1043456    c  W95 FAT32 (LBA)
/dev/sdb3       469507849   488386312     9439232    7  HPFS/NTFS/exFAT
/dev/sdb4   *    62637435   467411174   202386870    5  Extended
/dev/sdb5        62637498    66734009     2048256   82  Linux swap / Solaris             swap
/dev/sdb6        66734073   333059579   133162753+  83  Linux                            /data
/dev/sdb7       333059643   406444499    36692428+  83  Linux                            / oude 11.3 mag weg
/dev/sdb8       406444563   467411174    30483306   83  Linux                            / openSUSE 12.1 

1 ntfs
 4 extended
 5 swap
 6 ext4       /data
 7 ext4       /      opensuse 11.3 productie
 8 ext4       /      opensuse 11.4 testing (factory)
2 fat32       HP_TOOLS
3 ntfs        HP_RECOVERY

Boot & Disk

Using GRUB to boot multiple operating systems, the correct way

Often when you want to use more than one Operating System on your computer, you run into GRUB issues.

When you install an OS, or upgrade an installed OS, problems can occur when the new install overwrites your GRUB setup, leaving you with unbootable OS's. It is possible to correct these issues, but that is an annoying exercise that you will have to repeat every time you update stuff.

chainloader

In order to solve this issue for once and for all, the solution is to make 1 partition the 'master GRUB' partition. This GRUB will contain only so called 'chainloader' commands, pointing to all the 'child' partitions containing the OS's. Those partitions will contain the GRUB's from those OS's. So we create a 2 stage boot process:

In this scenario let all the OS's manage their own partitions, and THIS IS IMPORTANT: don't let them update our 'master GRUB' partition. They can update their own GRUB during updates etc.

MBR info

Note that the 'grub-install' method has the added protection of not inadvertantly overwriting your partition table if it had been modified since your last MBR backup.

This is because the 512byte MBR sector is actually two parts:

The first 446 bytes is the grub stage1 bootloader (or the windows bootloader after you’ve reinstalled windows and it “helpfully” overwrites grub).
The last 64 bytes is where your partition table is stored.

So, if you only want to backup the bootloader in the MBR, remember to change the bs=512 to bs=446.

What You Absolutely Need to Know

In order to use grub to boot a computer, you need to know the following:

The partition containing the kernel
Within that partition, the directory path and filename of the kernel
The partition containing /sbin/init

In addition, you might need the partition, path and filename of the initrd file, but usually this is not necessary with grub.

NOTE

I have seen cases in which a kernel would kernel panic without an initrd statement, and would boot with it. The interesting thing is, once I got it booting, I could remove the initrd statement, rerun grub's setup, and it would now boot without the initrd statement. If you get kernel panics and it isn't obvious why, don't hesitate to insert an initrd statement.

Now let's take a look at an example. Imagine a system in which /dev/hda1 is mounted as /boot, and /dev/hda9 is mounted as /. Within /boot the kernel filename is vmlinuz-i686-up-4GB. Now let's answer the four questions:

The partition containing the kernel = /dev/hda1, or (hd0,0) in grub-speak
Within that partition, the directory path and filename of the kernel = /vmlinuz-i686-up-4GB(Remember, /dev/hda1 is mounted directly to /boot, so it contains the kernel directly)
The partition containing /sbin/init is /dev/hda9

In that case, here are the grub commands you would input to boot that system:

grub> root (hd0,0)
grub> kernel /vmlinuz-i686-up-4GB root=/dev/hda9
grub> boot

The preceding is usually sufficient to boot a Linux box. The standalone root statement tells the partition containing the kernel. The kernel statement describes the path and filename, within the partition containing the kernel of the kernel. The argument to the root= argument to the kernel statement tells the partition containing /sbin/init, which of course turns out to be the root partition in the booted system.

Be careful of these duelling root keywords. The standalone one is the root as seen from grub, and contains the kernel. The argument to the kernel statement is the root as seen from the fully booted system, and contains /sbin/init.

Be careful also of where you use grub partition notation and where you use Linux partition notation. You use grub partition notation ((hd0,0)) everywhere except the root= argument to the kernel statement. In the root= argument you use the Linux partition notation. Note that in Linux notation, the drive starts with a for the first IDE port master, then b for the first IDE port slave, then c for the second IDE port master, and d for the second IDE port slave, on and on throughout your IDE ports. In Linux notation, the partition number within the drive starts with 1.

In grub partition notation, the first accessible hard drive is (hd0), the next accessible hard drive (even if it's on the 3rd, 4th or higher IDE port) is (hd1), and so forth. In grub partition notation, the partition number is zero based. Thus:

/dev/hda1 is the same partition as (hd0,0)

Occasionally you'll need to specify an initrd, although this is rare. If so, after the kernel statement and of course before the boot statement, insert the following:

initrd /initrd-i686-up-4GB.img

It's absolutely essential that if you do use an initrd statement, that the initrd file you reference must match the kernel you referenced earlier.

NOTE

Another documented way to boot from grub is to put the grub-root in the kernel statement itself instead of as a separate entity:

grub> kernel (hd0,0)/vmlinuz-i686-up-4GB root=/dev/hda9
grub> boot

If you do that, you'll need to also specify the grub root ((hd0,0)) on any initrd statement.

Booting Up Foreign Distros

Let's say you're a United States English speaker using grub to bust back into a Knoppix machine that lost its boot loader. The commands discussed previously would put you in Knoppix just fine, but the error messages and even the console keyboard would be German (Deutsch). If you wanted to boot up in American English, you'd add the argument lang=us to the kernel statement, like this:

grub> kernel (hd0,0)/vmlinuz-i686-up-4GB root=/dev/hda9 lang=us
grub> boot

grub> root (hd0,0)
grub> kernel /vmlinuz-i686-up-4GB root=/dev/hda9 lang=us
grub> boot

The Single Partition Configuration

The preceding example detailed a system with a dedicated /boot partition. Especially in these days of modern bioses that can boot past cylinder 1024, many people don't use a separate partition for /boot. Imagine if the root partition were /dev/hda1, and /boot was just another directory on that partition. In that case, here are the commands you'd use:

grub> root (hd0,0)
grub> kernel /boot/vmlinuz-i686-up-4GB root=/dev/hda1
grub> boot

The only difference is here the grub root is the same as the booted system root.

Having Grub Do Your Research For You

Often you know the partition containing the kernel, the kernel directory and name, and which partition mounts to root after boot. In that case booting Linux from grub is trivial.

Other times you're not so lucky. Like when you accidentally messed up LILO, or when you or someone else installed Windows, inadvertently overwriting the boot loader on the MBR. That's when you need grub the most, but that's also when you're least likely to know the partition containing the kernel, the partition that will ultimately be root, and the name of the kernel. Luckily, grub can help.

Your first step is to find the partition containing the kernel and the partition containing /sbin/init. Now type the following at the grub> prompt:

find /sbin/init

On a machine with three different Linux OS's installed, the answer would come back something like this:

grub> find /sbin/init

(hd0,8)
(hd0,11)
(hd1,11)
grub>

In the preceding example, you've found three different partitions containing /sbin/init:

Grub partition specification	Linux partition specification
(hd0,8)	/dev/hda9
(hd0,11)	/dev/hda12
(hd1,11)	/dev/hde12Note: I infer that hd1 maps to hde because on this particular machine there are two hard disks, one at hda and one at hde.

Next, find all partitions containing the kernel. Our first attempt assumes that at least one kernel will have filename vmlinuz.:

grub> find /vmlinuz

(hd0,0)
(hd0,11)
(hd1,11)
grub>

Then perform the same search for vmlinuz in a directory called /boot:

grub> find /boot/vmlinuz

(hd0,11)
(hd1,11)
grub>

Here we find only two of the three we found in the first attempt, because on this machine, (hd0,0) is mounted as /boot on one of the OS's.

Grub's find command is limited. It can find only regular files, not directories. Usually the entire directory path must be specified, although for some reason it finds a couple /boot/vmlinuz when you use find on /vmlinuz. Don't count on that behavior.

Another technique for finding info in grub is to use its file completion feature. Let's say you know the kernel is on (hd0,0) and the kernel file begins with vml. Press the tab key after issuing this partial command:

null (hd0,0)/vmlinuz

Grub performs file completion much like you see at a Linux command prompt.

grub> null (hd0,0)/vmlinuz

Possible files are: vmlinuz vmlinuz-2.6.3-7mdk vmlinuz-2.6.3-7mdk-i686-up-4GB
vmlinuz-i686-up-4GB
grub>

In the preceding, the word null is not a keyword, but instead a word chosen because it is not a keyword. Instead of "null", you could have used "whatever" or "bogus" or any other non-keyword. Once you get the list, you can complete a little more and then press tab again, just like at a bash prompt. By doing so you minimize the likelihood of transcription errors.

Occasionally grub won't easily give you all the necessary information. If you need more information than grub can conveniently provide, boot Knoppix. See Troubleshooters.Com's Knoppix Knowhow site for details.

Making a Full grub Boot Floppy

In this document's first article you created a simple Grub boot floppy without a filesystem. This is adequate to boot a computer, but not to install grub on the computer. Installing grub requires a boot floppy with grub on a filesystem. You can do that on any Linux box on which grub is installed. The following are the steps:

[root@mydesk root]# mkfs -t ext2 -c /dev/fd0u1440
[root@mydesk root]# umount /dev/fd0
[root@mydesk root]# umount /dev/fd0u1440
[root@mydesk root]# mkdir /mnt/test
[root@mydesk root]# mount /dev/fd0u1440 /mnt/test
[root@mydesk root]# mkdir -p /mnt/test/boot/grub
[root@mydesk root]# cp /boot/grub/stage1 /mnt/test/boot/grub
[root@mydesk root]# cp /boot/grub/stage2 /mnt/test/boot/grub
[root@mydesk root]# chmod a-w /mnt/test/boot/grub/stage2
umount /dev/fd0u1440
[root@mydesk root]# grub
grub> root (fd0)
grub> setup (fd0)
grub> quit
[root@mydesk root]#

You now have a bootable grub floppy with which you can boot a computer. One more thing should go on the floppy -- an example menu.lst. The menu.lst file is what brings up a "grub menu", and is vital for actually installing the grub bootloader on another computer. On the computer you need to boot, you can edit the menu.lst file to produce a grub menu on boot, and to actually install grub on the system. Note that the example menu.lst shoud NEVER be copied to the floppy before all the steps listed above this paragraph. Here is a typical session showing how to perform the copy. Note that once again, a mount and unmount must be performed.

[root@mydesk root]# mount /dev/fd0u1440 /mnt/test
[root@mydesk root]# cp -p /usr/share/doc/grub-doc-0.93/menu.lst /mnt/test/boot/grub/
stage1 stage2
[root@mydesk root]# cp -p /usr/share/doc/grub-doc-0.93/menu.lst /mnt/test/boot/grub/menu.lst.example
[root@mydesk root]# umount /dev/fd0u1440
[root@mydesk root]#

Installing grub From Floppy

Do not perform this exercise until you've practiced the earlier exercises. Knowledge of the operation of the grub command line interface is vital to creating and installing a menu driven grub.

WARNING

This exercise walks you through creating a floppy based grub boot floppy on a grub installation on computer 1, and then using that floppy to boot and configure grub on computer 2. It WILL wipe out any existing boot loader from computer 2. Hopefully it will replace that boot loader with grub, but there are no guarantees. In fact, this document addresses only Linux grub setups, so if you have Windows or BSD installed, this document cannot help you recover.Therefore, computer 2 MUST be an experimental computer whose data and OS you can afford to lose.

In the Making a Full grub Boot Floppy exercise you created a boot diskette with stage1, stage2, and an example menu.lst named menu.lst.example. Now it's time to use that floppy on another system. As mentioned in the preceding warning, that other system must be an experimental system whose boot loader you can afford to overwrite, possibly unsuccessfully.

Insert the full grub boot floppy in the experimental computer, shut the computer down in an orderly fashion, and reboot the computer. During BIOS boot, make sure the computer's first boot drive is the floppy.

It is very likely that the the other system has grub installed. To temporarily move its files aside, do the following:

mv /boot/grub /boot/orggrub
mv /sbin/grub /sbin/orggrub
mv /sbin/grub-install /sbin/orggrub-install

Basically, rename directory /boot/grub, and then rename all grub executables. In this way you're simulating a machine that has never had grub installed, yet you can "put back" the files with a couple more renames. However, it is not so simple to "put back" the MBR.

The basic procedure is as follows:

Boot from the grub floppy
Copy files from the floppy to /boot/grub on the hard disk
Configure /boot/grub/menu.lst for this computer
Reboot from floppy, and install grub

Boot from the grub floppy

grub> root (hd0,0)
grub> kernel /vmlinuz-i686-up-4GB root=/dev/hda9
grub> boot

As mentioned before, if booting to a foreign language distro, use the appropriate lang= kernel argument so that you can work in your native tongue. If everything went right, your experimental system is now booted.

Copy files from the floppy to /boot/grub on the hard disk

First make sure there's no /boot/grub. If there is, rename it, because you sure don't want to overwrite it just to perform this exercise.

Now perform the following commands:

mkdir /mnt/test
mount /dev/fd0u1440 /mnt/test
cp -Rp /mnt/test/boot/grub /boot

Configure /boot/grub/menu.lst for this computer

If you began this document as a grub newbie, the sample menu.lst that ships with grub would have been useless to you. What a difference a few exercises can make. You now know how to boot a computer from the grub> prompt. You know the difference between the grub root and the root directory seen by Linux after bootup. You know how to structure a grub kernel statement.

A menu.lst file is basically just the same list of commands you'd use at the grub> prompt, except that the boot command is not included. The example menu.lst has commands for installing operating systems from mach to Windows, and it even has an entry that installs grub on the system and another that changes the menu colors. All of that is extraneous. What you want to do is delete all the non-Linux stuff, and configure the Linux commands to match your experimental machine's kernel partition, Linux root partition, and kernel filename. The following is an example of such an edited example file saved as menu.lst:

A few notes are in order:

BE SURE to comment out the "fallback 1", because there is no choice 1, only a choice 0.
The default 0 statement simply points to which choice will run if the user makes no choice within the timeout period.
The lang= is unnecessary unless you're booting up a foreign distro and want to see it in your native language.
The root (hd0,0) needs to be adjusted to point to the partition containing the kernel.
The root=/dev/hda1 needs to be adjusted to point to the partition that will be mounted as root after booting. Note that it is expressed in Linux notation, not in grub notation.
The kernel statement points to the kernel relative to the grub root. In the preceding file it points to /boot/vmlinuz-2.6.7 in the Linux root partition. If Linux used a partition just for /boot, and that partition were /dev/hda1 as is typical of boot partitions, then the kernel would point to /vmlinuz-2.6.7 instead of /boot/vmlinuz-2.6.7, because the file would be on the root of the boot partition, and would become subservient to /boot only after the partition is mounted by Linux.
Grub's default menu file is /boot/grub/menu.lst. You can use a different file, but it would require jumping through quite a few hoops. Use the default.
None of this is new to you. You already did all of this at the grub> prompt in previous exercises.

Your /boot/grub/menu.lst file is now complete. If you created it correctly, you can now install grub from your floppy:

Reboot from floppy, and install grub

Insert your full grub boot floppy in the experimental machine, and reboot. The machine boots to the floppy, and you are presented with the grub> prompt. From there, installation is easy, assuming you've done the previous steps correctly:

grub> root (hd0,0)

grub> setup (hd0)
grub> reboot

The machine reboots again, and if you remove the floppy, you are presented a 1 item grub menu. This is what it looks like:

Pressing Enter on that item boots the kernel listed in your menu.lst. A few notes are in order:

Never remove the floppy while the floppy light is on
If you fail to remove the floppy before the reboot reaches the bootloader, it will boot from floppy again. In that case, just boot from floppy as you have in prior exercises, and then remove the floppy and reboot the machine to get the menu.

Summary

Thats it. You created a bootable floppy with stage1, stage2 and an example menu file, and used that floppy to install a grub bootloader on a machine. This is how you can take a machine with a blown bootloader and configure it to boot with grub.

However, your "grub installation" falls short. None of the grub executables are there:

grub
grub-floppy
grub-install
grub-md5-crypt
grub-reboot grub-terminfo

Also, many files normally placed in /boot/grub are not there:

device.map
e2fs_stage1_5
jfs_stage1_5
minix_stage1_5
reiserfs_stage1_5
xfs_stage1_5

The other thing you don't have is the documentation that usually comes in /usr/share/grub.Once you get the machine booting in a stable manner, you'll probably want to install grub from a package manager or by compiling the source. Be sure to back up your existing /boot/grub/menu.lst.

Working With a Fully Installed grub

You don't want to reboot to floppy just to get to a grub> prompt or to make changes to the bootloader on your MBR. Once you've installed the full grub package you don't need to. You can use grub-install to install a newly configured boot loader without resorting to your boot floppy. You can use grub to view the results of a new menu.lst.

After you've installed grub (or restored the original installation by undoing the renames), edit your menu.lst , insert a second entry that boots the 2.4 kernel instead of the 2.6, name each entry for its kernel number, and restore the. Here is the resulting file:

The preceding file has two boot choices: One for the 2.6 kernel (appropriately named 26) and one for the 2.4 kernel (appropriately named 24). It defaults to 26, but if for some reason 26 won't boot it falls back to 24.

Once the file is saved, perform the following command:

grub-install /dev/hda1

NOTE

To get the same result, you can also run grub, either from a command line or via a grub boot floppy, and perform the following command sequence:

grub> root (hd0,0)
grub> setup (hd0)

The preceding assumes that the kernel is located on (hd0,0), which is otherwise known as /dev/hda1, and that you're installing it to the MBR of the first hard disk, also known as /dev/hda.

The next time you boot your computer, you'll see choices 26 and 24, and they'll do the right thing, as follows:

If you want to view the menu without rebooting, try this command:

grub --config-file hd0, 0/boot/grub/menu.lst

In the preceding, note that you need to escape the parentheses of the grub partition because otherwise the shell will interpret those parentheses as special characters. Perform that command and you'll see the menu, which looks like the following:

However, you cannot actually boot from grub run on the command line.

Options from the grub menu

We all hope every item on the menu produces a clean boot. If not, you can either edit individual commands in a given menu choice by pressing the e key, or drop down to the grub command prompt by pressing the c key. Generally speaking, you edit commands if you suspect the menu choice is close to a correct configuration, and you drop directly to the grub command prompt if you suspect the menu item to be totally wrong, or if you need the facilities of grub's find command or its command completion feature.

Why Dedicated?

If you've ever had four or more Linux distros on one box, you already know the answer. Install a new distro, and you lose the ability to boot the others. Oh sure, some distros are nice enough to try to detect the others, but often label them wrong. Some distros don't even try, but assume they're the only distro on the disk. Some distros overwrite everything with LILO.

When installing, removing and tweaking multiple distros on a single box, what you really need is a single menu of all boots. Ideally, you don't want that menu associated with a distro, because it it were, it would be overwritten every time you remove or reinstall that distro.

This document walks you through creating a 1 MB hard disk partition containing the grub boot code and the grub menu for all your various boots. By keeping this one and only one menu up to date, you avoid the confusion of constantly having to re-import all other boots every time you install something new. Because the partition is only a single megabyte, you avoid the temptation to cannibalize it for something else.

Once your grub partition is in place, adding a distro is as simple as installing that distro, then mounting the grub partition, and adding the new distro's grub boot information (menu.lst) to the boot distro's menu.lst, and then resetting the hard disk MBR to point to the grub partition.

Creating Your Grub Partition

Unless you have very strange hardware with special booting needs, you can fit all Grub code into a 1MB partition. Create the partition with fdisk. The fdisk program is able to make tiny partitions not on cylinder boundaries. Other programs such as cfdisk round to the nearest cylinder boundary, thereby making this partition much bigger than it needs to be. For the same reason, use fdisk to create the partition after this one, to make sure that no space is left in the middle. From then on, assuming partitions are big, other partitioning programs are fine, because a few megabytes on one side or the other of a 2GB partition aren't significant.

If you're working with a very old bios with the ancient 1024 cylinder limitation, it should be very near the front of the disk (make it /dev/hda1). Remember, it's only a megabyte in size, so it's not going to "push out" any other partitions with this limitation (typically the /boot partition). As a practical matter, modern bios implementations don't have this limitation, so on modern machines you can put the grub partition anywhere.

Try to make sure to remember the device number (/dev/hda1) or whatever) housing the grub partition. However, if worst comes to worst, you can quickly see which partition using this command:

fdisk -lu /dev/hda

The preceding command prints out the number of sectors used by each partition. The Grub partition will be the smallest, because no modern operating system can be housed in a 1MB partition.

Once you've used fdisk to create the partition, format it with the mkfs.ext2 command. There's no use making this an ext3 partition, because it's small enough to fsck anyway.

Setting Up Your Grub Partition

This is simply a matter of copying files. At the very least, you must have these three files:

You can copy stage1 and stage2 from the /boot/grub directory of the booted OS, or any partition. However, if you have any unusual hardware or journaling file systems besides ext3, you'll need some of the "stage1_5" files. In such a case, ALL the stage files must be copied from the same source. On your currently booted distro, perform the following command:

locate stage

Here are some of the files that command shows on my Mandriva 2006 system:

/lib/grub/i386-mandriva/iso9660_stage1_5
/lib/grub/i386-mandriva/e2fs_stage1_5
/lib/grub/i386-mandriva/fat_stage1_5
/lib/grub/i386-mandriva/ffs_stage1_5
/lib/grub/i386-mandriva/reiserfs_stage1_5
/lib/grub/i386-mandriva/jfs_stage1_5
/lib/grub/i386-mandriva/minix_stage1_5
/lib/grub/i386-mandriva/stage2_eltorito
/lib/grub/i386-mandriva/stage1
/lib/grub/i386-mandriva/stage2
/lib/grub/i386-mandriva/ufs2_stage1_5
/lib/grub/i386-mandriva/vstafs_stage1_5
/lib/grub/i386-mandriva/xfs_stage1_5

I could copy ALL those files to the grub partition, or just stage1 and stage2.Here's how I'd do it:

mkdir /mnt/test
mount /dev/hda1 /mnt/test
mkdir -p /mnt/test/boot/grub
cp -p /lib/grub/i386-mandriva/* /mnt/test/boot/grub

Note that on my system, a du command reveals that all those files consume only 376KB, so there's plenty of room.

The only thing left to do is create /mnt/test/boot/grub/menu.lst.

If possible, find a partition whose menu.lst contains choices for booting several different partitions. If the partition having such a menu.lst were /dev/hda7 and your Grub partition were /dev/hda1, and you mounted them as /mnt/hda1 and /mnt/hda7, you'd perform the following command:

cp /mnt/hda7/boot/grub/menu.lst /mnt/hda1/boot/grub

Now that you have a starter for menu.lst, edit the file and pull in the info from menu.lst files from other partitions, straight into your editor. In each case, delete out the new global information and keep only the info for each new boot. Be careful that you don't get two different choices for essentially the same boot.

Occasionally, on LILO-only distributions, you might find no menu.lst file. In that case, import that distro's /etc/lilo.conf and translate LILO menu info to Grub menu info. It's not difficult if you do it slowly.

As you edit the file, make different versions so that if it doesn't work you can try a simpler version.

Making it Happen at Boot Time

You've made your menu.lst on your Grub partition, complete with boot info from every distro on the system. The trouble is, that menu file won't be consulted at boot time. To get the boot to consult it, you need to tell the hard disk's MBR to consult your Grub partition. Assuming your Grub partition is /dev/hda1, here's how you do it:

# grub
grub> root (hd0,0)
grub> setup (hd0)
grub> quit
# reboot

In the preceding command sequence, the first runs the grub executable. The second sets the root to (hd0,0), which is grub-speak for /dev/hda1. The third command tells grub to install code on the MBR that looks to /dev/hda1 for boot code. The fourth command terminates the grub command, and the fifth reboots the computer. When the computer reboots, if all went well you should see the menu you set up on your grub partition. If not, troubleshoot. If need be, bust back into the machine using the Grub techniques on the Grub From the Ground Up page, using either a Grub floppy or a Knoppix CD.

Adding a New Distro

This is where you start to see the real benefit of the dedicated Grub partition. In the bad old days, installing one more distro on your hard disk would necessitate a long and difficult search for other boots, culminating in voluminous troubleshooting. Now you simply edit your Grub partition's menu.lst and add the boots (not the globals) from the latest distro addition. Booted into the latest distro, signed in as root, it would go something like this:

Mount /dev/hda1 /mnt/test
cd /mnt/test/boot/grub
cp -p menu.lst menu.lst.bup
vim menu.lst
Within vim:
1. G to get to the bottom
2. o <Enter><Enter><Enter><Enter> (in order to open a space)
3. :r /boot/grub/menu.lst
  1. NOTE: If the new version had /etc/lilo.conf and no /boot/grub/menu.lst, you might need to read in lilo.conf and then translate. It's not particularly difficult.
4. Delete all global info brought in by the new menu.lst, and also any boots you don't need.
5. :wq
Run Grub, and within Grub do this:
1. grub> root (hd0,0)
2. setup (hd0)
Reboot the computer, and your list should include all old boots as well as those brought in by the new boots.

The Case for Doing this On Every Hard Disk

A tiny dedicated Grub partition makes it easy to include a dedicated Knoppix partition, along with anything else you might need on the hard drive.

I find life much easier with a Knoppix partition on every hard disk I own. On dedicated data drives, it allows me to grab my data if my system drive should become defunct. It allows easy transfer of data between an old data disk and its replacement, either with the old disk still in the production computer and the new disk in a spare computer, or the new disk in the production computer and the old disk in the spare computer (in other words, you forgot to do it before shutting down the production computer with the old disk -- it happens).

You never know when somebody, some time, will dual or tri-boot the system. Or install six different OS's on it. Having a dedicated Grub partition makes such events much easier to handle.

Boot & Disk

Install Linux on an SSD

When you want to install Linux to an SSD, there are a lot of things to think about. This wiki post will look at all those things.

1. start your partitons on a 4k boundary

Because SSD's use 'blocks' of 4k in size, you must line up the partition boudaries on a block boudary for the best performance.

In fact, it is even better to start your partitons on a 512k block boundary. To do that, start fdisk with the parameters '-H 32 -C 32 -c' and create a completely new partitiontable.

fdisk -H 32 -C 32 -c

'''Note: Intel SSD's uses smaller blocks so use a different command:

fdisk -H 16 -S 16 -c'''

2. Optimize I/O

To adjust the stripe width and stride of an EXT4 filesystem, use this command to create a new filesystem:

mkfs.ext4 -E stride=128,stripe-width=128 /dev/sdaX

where sdaX is of course your ssd partition.

To tune an already existing filesystem, you can use:

tune2fs -E stride=128,stripe-width=128 /dev/sdaX

Intel users should use the value 32 for the stripe width

Use a different scheduler

You can also use a different i/o scheduler, the deadline scheduler, because the one that is used as a standard in the kernel is optimized for magnetic hard drives.

Modify /etc/rc.local (or /etc/rc.d/boot.local on openSUSE) and insert just before 'exit 0'

echo deadline > /sys/block/sda/queue/scheduler

3. Use TRIM

3.1 first, check if your SSD supports the TRIM command

hdparm -I /dev/sda | egrep "Enabled|TRIM"

3.2 second, your kernel needs to be > 2.6.33

uname -r

3.3 without special mount options, TRIM will not be used. To enable TRIM, use the 'discard' mount option in /etc/fstab.

/dev/sdaX / ext4 discard,etc. etc. etc.......

Also, you can use the noatime option to minmize the wear on the disk, so that would make the fstab line:

/dev/sdaX / ext4 noatime,discard,etc. etc. etc.......

Re-using an SSD

On occasion, users may wish to completely reset an SSD's cells to the same virgin state they were manufactured, thus restoring it to its factory default write performance. Write performance is known to degrade over time even on SSDs with native TRIM support.

When you want to reformat an SSD, it is best to completely wipe the disk. That way the algorithm will also be reset to get the best possible performance in your new configuration. To do that, you can use hdparm XXXX to be added.. XXXXXXX

Further performance enhancements

To increase system performance and to eliminate unnecessary write actions to the SSD, you can use a RAM disk.

Add this line to /etc/fstab:

tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0

Boot & Disk

Btrfs

list available devices

btrfs device scan

list available filesystems

btrfs filesystem show

create volume

mkfs.btrfs /dev/sdX

stripe across devices (mirror metadata + stripe data)

mkfs.btrfs /dev/sda /dev/sdb /dev/sdc

stripe across devices (stripe metadata + stripe data)

mkfs.btrfs -m raid0 /dev/sda /dev/sdb /dev/sdc

mirror across devices (mirror metadata + mirror data)

mkfs.btrfs -m raid10 -d raid10 /dev/sda /dev/sdb /dev/sdc

create subvolume

btrfs subvolume create /mnt/nonsense

where /mnt is the mountpoint of the btrfs volume

list subvolumes

btrfs subvolume list /mnt

create snapshot

mkdir /mnt/snapshots
btrfs subvolume snapshot /mnt/nonsense /mnt/snapshots/vrijdag

add a new device

btrfs device add /dev/sda8 /mnt

remove a device

btrfs delete /dev/sdb /mnt

free space:

btrfs fi df /

show snapshots

snapper list

delete all snapshots

for i in {1..20000}; do snapper delete $i; done

2015:

balance om ruimte vrij te maken:

btrfs balance start /tmp/bla

als geen ruimte vrij:

btrfs balance start -musage=50 -dusage=50 /tmp/bla

btrfs fi show /tmp/bla
btrfs fi df /tmp/bla

Onderhoud:

btrfs scrub is used to scrub a btrfs filesystem, which will read all data and metadata blocks from all devices and verify checksums. Automatically repair corrupted blocks if there’s a correct copy available.

btrfs scrub start -B /tmp/bla
fstrim /tmp/bla

for conventiaonal HD's:

btrfs filesystem defragment /tmp/mnt/bla

recursief een directory:

btrfs filesystem defragment -r /tmp/mnt/bla

Boot & Disk

Fix grub

Boot from live dvd OR choose RESCUE from boot USB (graphical desktop is ok)
use gparted to find boot partition
mount /dev/sdX /mnt

mount /dev/sda5 /mnt

mount –-bind /dev /mnt/dev
mount –-bind /proc /mnt/proc
mount –-bind /sys /mnt/sys
chroot /mnt
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-install /dev/sda
chtrl-d 'exit'
reboot

grub edit

F2 into grub CLI

insmod ntldr

insmod ntfs

insmod chain

set root=(hd0,msdos1)

chainloader +1

boot

Boot & Disk

Usb drives

USB Drives Overzicht
USB Stick	Capacity	speed	Contents	Remarks
Voyager	8GB	24	Windows 10 home??	Example
RedHat	16	7	Leap 15.5	X86_64
WitteSticker	2	10-15	GPartEd live 2024	20220202
SanDisk	32GB	130	KDE Live! TW	bootable
Ictivity	3.8GB	?	audioanalyzer files	FAT16
Dupaco	4GB	3	HANTEK
openSUSE	4GB	26	leeg	Example
Example	Example	Example	Example	Example
Example	Example	Example	Example	Example
Example	Example	Example	Example	Example

drive (capacity): speed    current image
voyager(8): 24           win10 home?
redhat(16): 7            opensuse tumbleweed       20180723
opensuse(4): 26          GPARTED Live              20200820
opensusekoord:           supergrubbootdisk
dupaco(4): 3             opensuse Leap 15.0 NET
Sandisk(32): 130         kde live TW
Emtec:                     Ansible cursus dump
wittesticker(2): 10-15


measured on Envy.


pi3:/etc/sysconfig/network # hdparm -t /dev/mmcblk0

/dev/mmcblk0:
 Timing buffered disk reads:  66 MB in  3.09 seconds =  21.39 MB/sec

pi3:/etc/sysconfig/network # hdparm -t /dev/sdb    <-- this is the redhat usb disk

/dev/sdb:
 Timing buffered disk reads:  38 MB in  3.03 seconds =  12.55 MB/sec

pi3:/etc/sysconfig/network # hdparm -t /dev/sda  <--- dit is de snelle sandisk

/dev/sda:
 Timing buffered disk reads:  88 MB in  3.05 seconds =  28.82 MB/sec

Boot & Disk

GRUB

Published on Linux.com | The source for Linux information (https://www.linux.com)

Home > How to Rescue a Non-booting GRUB 2 on Linux

How to Rescue a Non-booting GRUB 2 on Linux [1]

Submitted by cschroder [2] on

Once upon a time we had legacy GRUB, the Grand Unified Linux Bootloader version 0.97. Legacy GRUB had many virtues, but it became old and its developers did yearn for more functionality, and thus did GRUB 2 come into the world.

GRUB 2 is a major rewrite with several significant differences. It boots removable media, and can be configured with an option to enter your system BIOS. It's more complicated to configure with all kinds of scripts to wade through, and instead of having a nice fairly simple /boot/grub/menu.lst file with all configurations in one place, the default is /boot/grub/grub.cfg. Which you don't edit directly, oh no, for this is not for mere humans to touch, but only other scripts. We lowly humans may edit /etc/default/grub, which controls mainly the appearance of the GRUB menu. We may also edit the scripts in /etc/grub.d/. These are the scripts that boot your operating systems, control external applications such as memtest and os_prober, and theming./boot/grub/grub.cfg is built from /etc/default/grub and /etc/grub.d/* when you run the update-grub command, which you must run every time you make changes.

The good news is that the update-grub script is reliable for finding kernels, boot files, and adding all operating systems to your GRUB boot menu, so you don't have to do it manually.

We're going to learn how to fix two of the more common failures. When you boot up your system and it stops at the grub> prompt, that is the full GRUB 2 command shell. That means GRUB 2 started normally and loaded the normal.mod module (and other modules which are located in /boot/grub/[arch]/), but it didn't find your grub.cfg file. If you see grub rescue> that means it couldn't find normal.mod, so it probably couldn't find any of your boot files.

How does this happen? The kernel might have changed drive assignments or you moved your hard drives, you changed some partitions, or installed a new operating system and moved things around. In these scenarios your boot files are still there, but GRUB can't find them. So you can look for your boot files at the GRUB prompt, set their locations, and then boot your system and fix your GRUB configuration.

GRUB 2 Command Shell

The GRUB 2 command shell is just as powerful as the shell in legacy GRUB. You can use it to discover boot images, kernels, and root filesystems. In fact, it gives you complete access to all filesystems on the local machine regardless of permissions or other protections. Which some might consider a security hole, but you know the old Unix dictum: whoever has physical access to the machine owns it.

When you're at the grub> prompt, you have a lot of functionality similar to any command shell such as history and tab-completion. The grub rescue> mode is more limited, with no history and no tab-completion.

If you are practicing on a functioning system, press C when your GRUB boot menu appears to open the GRUB command shell. You can stop the bootup countdown by scrolling up and down your menu entries with the arrow keys. It is safe to experiment at the GRUB command line because nothing you do there is permanent. If you are already staring at the grub> or grub rescue>prompt then you're ready to rock.

The next few commands work with both grub> and grub rescue>. The first command you should run invokes the pager, for paging long command outputs:

There must be no spaces on either side of the equals sign. Now let's do a little exploring. Type ls to list all partitions that GRUB sees:

grub> ls

(hd0) (hd0,msdos2) (hd0,msdos1)

What's all this msdos stuff? That means this system has the old-style MS-DOS partition table, rather than the shiny new Globally Unique Identifiers partition table (GPT). (See Using the New GUID Partition Table in Linux (Goodbye Ancient MBR) [3]. If you're running GPT it will say (hd0,gpt1). Now let's snoop. Use the ls command to see what files are on your system:

grub> ls (hd0,1)/

lost+found/ bin/ boot/ cdrom/ dev/ etc/ home/ lib/

lib64/ media/ mnt/ opt/ proc/ root/ run/ sbin/

srv/ sys/ tmp/ usr/ var/ vmlinuz vmlinuz.old

initrd.img initrd.img.old

Hurrah, we have found the root filesystem. You can omit the msdos and gpt labels. If you leave off the slash it will print information about the partition. You can read any file on the system with the cat command:

grub> cat (hd0,1)/etc/issue

Ubuntu 14.04 LTS \n \l

Reading /etc/issue could be useful on a multi-boot system for identifying your various Linuxes.

Booting From grub>

This is how to set the boot files and boot the system from the grub> prompt. We know from running the ls command that there is a Linux root filesystem on (hd0,1), and you can keep searching until you verify where /boot/grub is. Then run these commands, using your own root partition, kernel, and initrd image:

grub> set root=(hd0,1)

grub> linux /boot/vmlinuz-3.13.0-29-generic root=/dev/sda1

grub> initrd /boot/initrd.img-3.13.0-29-generic

grub> boot

The first line sets the partition that the root filesystem is on. The second line tells GRUB the location of the kernel you want to use. Start typing /boot/vmli, and then use tab-completion to fill in the rest. Type root=/dev/sdX to set the location of the root filesystem. Yes, this seems redundant, but if you leave this out you'll get a kernel panic. How do you know the correct partition? hd0,1 = /dev/sda1. hd1,1 = /dev/sdb1. hd3,2 = /dev/sdd2. I think you can extrapolate the rest.

The third line sets the initrd file, which must be the same version number as the kernel.

The fourth line boots your system.

On some Linux systems the current kernels and initrds are symlinked into the top level of the root filesystem:

$ ls -l /

vmlinuz -> boot/vmlinuz-3.13.0-29-generic

initrd.img -> boot/initrd.img-3.13.0-29-generic

So you could boot from grub> like this:

grub> set root=(hd0,1)

grub> linux /vmlinuz root=/dev/sda1

grub> initrd /initrd.img

grub> boot

Booting From grub-rescue>

If you're in the GRUB rescue shell the commands are different, and you have to load the normal.mod andlinux.mod modules:

grub rescue> set prefix=(hd0,1)/boot/grub

grub rescue> set root=(hd0,1)

grub rescue> insmod normal

grub rescue> normal

grub rescue> insmod linux

grub rescue> linux /boot/vmlinuz-3.13.0-29-generic root=/dev/sda1

grub rescue> initrd /boot/initrd.img-3.13.0-29-generic

grub rescue> boot

Tab-completion should start working after you load both modules.

Making Permanent Repairs

When you have successfully booted your system, run these commands to fix GRUB permanently:

# update-grub

Generating grub configuration file ...

Found background: /usr/share/images/grub/Apollo_17_The_Last_Moon_Shot_Edit1.tga

Found background image: /usr/share/images/grub/Apollo_17_The_Last_Moon_Shot_Edit1.tga

Found linux image: /boot/vmlinuz-3.13.0-29-generic

Found initrd image: /boot/initrd.img-3.13.0-29-generic

Found linux image: /boot/vmlinuz-3.13.0-27-generic

Found initrd image: /boot/initrd.img-3.13.0-27-generic

Found linux image: /boot/vmlinuz-3.13.0-24-generic

Found initrd image: /boot/initrd.img-3.13.0-24-generic

Found memtest86+ image: /boot/memtest86+.elf

Found memtest86+ image: /boot/memtest86+.bin

done

# grub-install /dev/sda

Installing for i386-pc platform.

Installation finished. No error reported.

When you run grub-install remember you're installing it to the boot sector of your hard drive and not to a partition, so do not use a partition number like /dev/sda1.

But It Still Doesn't Work

If your system is so messed up that none of this works, try the Super GRUB2 live rescue disk [4]. The official GNU GRUB Manual 2.00 [5] should also be helpful.

Tutorial Category:

Tutorials [6]

Source URL: https://www.linux.com/learn/how-rescue-non-booting-grub-2-Linux

Links:[1] https://www.linux.com/learn/how-rescue-non-booting-grub-2-Linux[2] https://www.linux.com/users/cschroder[3] https://www.linux.com/learn/tutorials/730440-using-the-new-guid-partition-table-in-linux-good-bye-ancient-mbr-[4] http://www.supergrubdisk.org/[5] https://www.gnu.org/software/grub/manual/grub.html[6] https://www.linux.com/tutorials/category/tutorials

Boot & Disk

RAID

RAID vergroten

Ter referentie, hier enkele veel gebruikte commando's:

 # cat /proc/mdstat: geef status, ook rebuild voortgang
 # mdadm -D /dev/md[012]: detail over RAID 
 # mdadm - -stop /dev/md2, eerst umount /dev/md2: om een RAID te stoppen
 # mdadm - -assemble - -scan: assemble all raids
 # mdadm - -assemble /dev/md1 /dev/sda2 /dev/sdb2 /dev/sdc2: assemblatie
 # mdadm -f /dev/sda1: fail deze partitie (mark as FAULTY)
 # mdadm -r /dev/sda1: remove deze partitie
 # mdadm - -re-add /dev/md1 /dev/sdb2 *(veel gebruikt): om een recently removede schijf weer toe te voegen
doe dit nadat de schijf er weer is.
 # mdadm -a /dev/mdx: hot add listed devices
 # mdadm - -stop /dev/md2: stop de array
 # Mdadm -A - -scan: start the arrays

De volgende stappen zijn nodig:

0. zet de automatische reboot van de server UIT!

1. Selekteer een disk en verwijder hem van de array. Voer de 2 commando’s achter elkaar uit:

# mdadm -f /dev/md1 /dev/sdd1 - fail device
# mdadm -r /dev/md1 /dev/sdd1 - remove device

Het laatste commando doet het niet als drive al weg is, dus doe het voor loskoppelen. Daarna de oude drive loshalen en sluit de nieuwe drive aan.

2. partitioneer de nieuwe drive naar de volledige grootte. Laat op linux FS staan, dit werkt ook – PvdM 2017

# fdisk /dev/sdd

g: create new GPT partition table n: new partition en 4x enter w: write and quit:

3. Voeg de nieuwe disk aan het array toe:

# mdadm --add /dev/md1 /dev/sde1 - add device, start immediate rebuild

4. Daarna kun je het hele proces in de gaten houden met:

# watch cat /proc/mdstat
# mdadm –detail /dev/md1

5. Wacht tot de resync klaar is. Hierna moet de stappen 1-4 herhalen voor elke schijf. 6. Hierna kun je de ruimte op het array vergroten:

# mdadm --grow /dev/md0 --size=max - change the size of an active array

Dit duurt even, de extra (lege haha) ruimte wordt nu gesyncd.

7. Het array bevat nu 1 disk met alle beschikbare ruimte. 8. Nu de onderliggende partitie is vergroot, moet je je filesystem resizen. Maar eerst wil je een filesystemcheck uitvoeren. Dit kan niet als het gemount is.

# umount [mountpoint]
# fsck /dev/md1

9. For an ext2/ext3/ext4 filesystem:

# resize2fs /dev/md1

Na het wegvallen van 1 harddisk

# mdadm /dev/md1 --add /dev/sdX

en dan werkt het weer

Boot & Disk

Swap file on BTRFS

Then start GParted or another disk management app, delete the swap partition, expand your partitions (BTRFS can be grown online, no need for reboot).
Create a subvolume to store your swap. Then create a swap file:

https://wiki.archlinux.org/title/Swap#Swap_file

You may wish to disable Copy On Write on your swap file.

Boot & Disk

Test harddisk for bad blocks

# badblocks -e 10 -s -w /dev/mmcblk0

-e max bad block count
Specify a maximum number of bad blocks before aborting the test. The default is 0, meaning the test will continue until the end of the test range is reached.

-s show progress

-w use write mode

-n Use non-destructive read-write mode.
By default only a non-destructive read-only test is done. This option must not be combined with the -w option, as they are mutually exclusive.

Pass completed, X bad blocks found (R/W/C errors)

The three numbers in the parentheses correspond to:

Read errors: Errors encountered while reading from the disk.
Write errors: Errors encountered while writing to the disk.
Corruption (Compare) errors: Errors where the data read back from the disk does not match what was written (i.e., data corruption detected during verification)

# smartctl -t long /dev/device

remember no smart on sd cards

Boot & Disk

Partitions-hupsie

hd0

476GB

P1 260MB EFI
P2 16MB MS RESERVED
P3 134GB win
P5 100GB /
P6 120GB /home
P7 100GB /data
P8 16GB swap
659MB herstelpartitie

Boot & Disk

Grub2: boot USB from GRUB prompt

When the PC will not boot from USB, you can use GRUB2 to boot from the USB.

grub> ls
(hd0) (hd0,msdos1) ...
grub> set root=hd0,msdos1
grub> chainloader /efi/boot/bootx64.efi
grub> boot

The above is of course an example, every situation will be different. But the idea is the same.

Boot & Disk

Best way to use GRUB in a multiboot system

default 0
timeout 8
gfxmenu (hd0,1)/boot/message
title Suse
   rootnoverify (hd0,1)
   chainloader +1
title Arch
   rootnoverify (hd0,2)
   chainloader +1
title Debian
   rootnoverify (hd0,6)
   chainloader +1

Boot & Disk

How to resue your system when the kernel won't boot

SUSE Broken? Don’t fear the chroot !

Posted by johnlange on September 22, 2009 06:45 pm under SUSE, Tech Tip

SUSE hasn’t let me down very often but recently I had a bad experience while applying some updates to an OpenSUSE laptop. There were quite a few updates so I undocked the laptop so I could relax while they downloaded.

For reasons that I have not yet resolved, the wirless networking became unstable and as a result, the updates had to be aborted.

Unfortunately, a new kernel was part of the updates and when the laptop rebooted it was in a bad state. X windows wouldn’t start and critically, there were no network drivers for the new kernel. To make matters worse, OpenSUSE does not keep the old kernels in /boot (why is that?) so there was nothing to fall back on.

With nothing left to do, it was time to try rescue mode and in a few short steps I had the system fully working again. Here is what I did:

Step 1: boot to rescue mode (duh).

Step 2: mount your hard disk partitions under /mnt in the same layout they would be normally. For example:

# mount /dev/sda2 /mnt
# mount /dev/sda1 /mnt/boot
… etc.

Step 3: Next we need to make sure we have acess to all the important system resources.

# mount --bind /proc /mnt/proc
# mount --bind /sys /mnt/sys
# mount --bind /dev /mnt/dev

Step 4: We’re ready to chroot into our new environment.

# chroot /mnt

Step 5: We are now running on our system just as if we had booted to it and we can perform repairs. In my case all I needed to do was complete the updates:

# zypper up

I rebooted and everything was back to normal.

Boot & Disk

How to restore MBR on harddisk

MBR / Hard Disk Layout

The MBR is the very first part of your hard disk stores the boot loader and the partition table.

Basically it comes to this: The MBR is 512 bytes . . the first 446 bytes are for the boot loader, and the bytes from 446 to 512 are for the partition table. If you delete the full 512 bytes you will not only delete the boot loader but you will have lost the partition table as well . . . so . . be careful !

If you back up the MBR to floppy you do:

# dd if=/dev/hda of=/dev/fd0/boot.mbr bs=512 count=1

Once the disk is clean and you want to restore:

# dd if=/dev/fd0/boot.mbr of=/dev/hda bs=512 count=1

This will restore the original partition table and the boot loader you had in the MBR.

Now, if you only want to clear the boot loader part ( and keep the partition table intact ) you do:

# dd if=/dev/zero of=/dev/hda bs=446 count=1

Boot & Disk

How to resque a hanging linux

key	function
r	Remove X's control over the keyboard
e	End all processes except init (SIGTERM)
i	Kill all processes except init (SIGKILL)
s	Write diskcache to disk
u	Remount all mounted filesystems read-only
b	Reboot!

Hardware & Machines

Systems

Hardware & Machines

Lap088

Hardware & Machines

Akoya

Hardware & Machines

Partitions akoya

Per 1 mei 2010

/dev/sda 75GB

1 ntfs    16GB windows/C boot
3 swap    1GB
4 extndd 50GB
 8 ext3   16GB /       SnowyWindOSX 
 5 ext4   16GB /       mounted on /data
 6 ext2  200MB /boot   <============================
 7 ext4   18GB         jolicloud
2 FAT32    8GB  windows/D recover

Bovenstaand is in volgorde op de disk.

Hardware & Machines

Zbox

Partities

OCZ Onyx Series SATA II 32 GB SSD

1. /boot         200MB
2. swap          1GB
3. Xbmc          8GB
4. ext. part.
 5. XbmcFreak    6GB
 6. OS 11.4      8GB /
 7. OS 12.1      6GB /
 8.

(parted) print                                                            
Model: ATA OCZ-ONYX (scsi)
Disk /dev/sda: 32.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type      File system     Flags
 1      1049kB  206MB   204MB   primary   ext2            type=83
 2      206MB   1283MB  1078MB  primary   linux-swap(v1)  type=82
 3      1283MB  9870MB  8587MB  primary   ext4            type=83
 4      9871MB  32.0GB  22.1GB  extended                  boot, lba, type=0f
 5      9871MB  16.3GB  6439MB  logical   ext4            type=83
 6      16.3GB  24.9GB  8587MB  logical   ext4            type=83
 7      24.9GB  32.0GB  7105MB  logical   ext4            type=83

#fdisk -l

Disk /dev/sda: 32.0 GB, 32017047552 bytes
255 heads, 63 sectors/track, 3892 cylinders, total 62533296 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000dc891

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048      401407      199680   83  Linux
/dev/sda2          401408     2506751     1052672   82  Linux swap / Solaris
/dev/sda3         2506752    19277823     8385536   83  Linux
/dev/sda4   *    19279870    62509055    21614593    f  W95 Ext'd (LBA)
/dev/sda5        19279872    31856639     6288384   83  Linux
/dev/sda6        31858688    48629759     8385536   83  Linux
/dev/sda7        48631808    62509055     6938624   83  Linux

Hardware & Machines

Hpspil

partitions hpspil

Hardware & Machines

Partitions hpspil

/dev/sda 112 gb

1 105 gb ntfs
2 7 gb ntfs

1 65 GB ntfs c: windows
4 38 GB ext4 /
3 1 GB swap
2 7 GB ntfs resque

Hardware & Machines

Cpu3972

Hardware & Machines

Samsie

Hardware & Machines

Partitions samsie

sda1: swap
sda2: /      btrfs  UUID=.....3858
sda3: /home  xfs    UUID=.....ce29





OLD:
1. systeem?        100mb ntfs (of 22gb?)
2. c:              230GB ntfs
3. d:              343GB ntfs
4. systeempartitie 22GB  ntfs


sda1 100MB ntfs system
sda2 230GB ntfs c: mounted on /mnt/sda2
sda3 343GB extended partition
sda4 22GB  ntfs  samsung recovery
sda5 30GB  ext4  /    root partition
sda6 2GB   swap
sda7 311GB fat32 /data

Hardware & Machines

WeTab

Hardware & Machines

Partitions cpu3972

cpu3972:/home/pvdm # fdisk -l

Disk /dev/sda: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7e407e40

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1              63    81931499    40965718+   7  HPFS/NTFS/exFAT
/dev/sda2        81931500   123877213    20972857    7  btrfs                      <== openSUSE 12.1
/dev/sda3   *   123877376   488396799   182259712    f  W95 Ext'd (LBA)
/dev/sda5       123879424   124278783      199680   83  Linux                      <== boot; MBR wijst ook hier naar toe. MASTER grub
/dev/sda6       124280832   128471039     2095104   82  Linux swap / Solaris
/dev/sda7       128473088   170418175    20972544   83  Linux                      <== openSUSE 11.4 /
/dev/sda8       170420224   488376319   158978048   83  Linux                      <== data

cpu3972:/home/pvdm # mount | grep sda | sort
/dev/sda1 on /windows/C type fuseblk (rw,nosuid,nodev,noexec,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)
/dev/sda2 on /windows/D type fuseblk (rw,nosuid,nodev,noexec,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)
/dev/sda5 on /boot type ext2 (rw,relatime,user_xattr,acl,barrier=1) <== NOT MEER GEMOUNT
/dev/sda7 on / type ext4 (rw,noatime,user_xattr,acl,barrier=1,data=ordered)
/dev/sda8 on /data type ext4 (rw,noatime,user_xattr,acl,barrier=1,data=ordered)

Hardware & Machines

Amd64

Per 20220917:

10:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 730] (rev a1)

Hardware & Machines

Hwinfo akoya

wireless: RT2860 Ralink

Hardware & Machines

Specs samsie

TECHNISCHE SPECIFICATIES VAN DE SAMSUNG RV720-S02NL

Product

Fabrikantcode

Artikelnummer

Garantie

Garantietype

Processor

Processor

Processornummer

Processorkernen

Kloksnelheid

Geheugen

Intern RAM-geheugen

Type geheugen

Harde schijf

Opslagcapaciteit

Type harde schijf

Hardeschijfsnelheid

Aantal schijven (intern)

Beeldscherm

Schermdiagonaal

Resolutie

Schermtype

Type beeldscherm

High Definition

Touchscreen

Videokaart

Videokaartfamilie

Type videokaart

Videogeheugen

Optische drive

DVD-speler/brander

Blu-ray

Communicatie

Ethernetsnelheid

Wi-Fi

Wifi-standaard

Bluetooth

RJ11 Modem

Ingebouwde 3G modem

Audio

Microfoon intern

Camera

Webcam

Webcam resolutie

Bediening

Toetsenbordindeling

Backlit toetsenbord

Numeriek keypad

Touchpad

Multi-Touch

Aansluitingen

e-SATA

ExpressCard

HDMI

Aantal USB-poorten

USB-versie

VGA

3,5 mm Audio

3,5 mm Microfoon

Geheugenkaartlezer

Beveiliging

Trusted Platform Module (TPM)

Vingerafdruksensor

Accu

Accu

Aantal cellen

Software

Besturingssysteem

Versie besturingssysteem

Taal besturingssysteem

Fysieke eigenschappen

Lengte

Breedte

Dikte

Gewicht

Kleur

Overige

Merk

Gebruik

Hardware & Machines

Specs lap088

Specificaties HP EliteBook 8530p

	Algemeen
	Merk
	Productnaam
	Productcode
	EAN code
	Toegevoegd op

	Processor / Chipset / Geheugen	Zoek producten op basis van specificatie
	Processor - Type	Intel Core 2 Duo P8800
	Processor - Klokfrequentie	2,66 GHz
	Chipset	Intel PM45
	Geheugen capaciteit	2 GB
	Type geheugen	DDR2
	SODIMM sloten	2
	Geheugen uitbreidbaar tot	8 GB

	Graphics	Zoek producten op basis van specificatie
	Type videokaart	ATI Mobility Radeon HD 3650
	Videogeheugen - Hoeveelheid	256 MB
	Scherm - Afmeting	15,4 inch
	Scherm - Resolutie	1680x1050
	DVI-uitgang
	VGA-uitgang
	HDMI-uitgang

	Opslag	Zoek producten op basis van specificatie
	Harddisk/SSD - Capaciteit	250 GB
	Harddisk/SSD - Toerental	7.200 rpm
	DVD branden

	Connectiviteit	Zoek producten op basis van specificatie
	LAN - Snelheid	1.000 Mbit/s
	WLAN - Standaard	802.11n
	Modem
	Bluetooth
	Infrarood
	HSDPA/UMTS

	Audio	Zoek producten op basis van specificatie
	Geïntegreerde microfoon

	Aansluitingen	Zoek producten op basis van specificatie
	PC-Card Type II	0
	Smartcard
	USB 2.0	4
	FireWire - Totaal	1
	eSATA	1
	Aansluiting dockingstation

	Overige Hardware	Zoek producten op basis van specificatie
	Ingebouwde webcam

	Maten en gewichten	Zoek producten op basis van specificatie
	Gewicht - Incl. accu	2,86 kg
	Afmeting - Breedte	35,5 cm
	Afmeting - Diepte	26,3 cm

	Software	Zoek producten op basis van specificatie
	Besturingssysteem	Microsoft Windows Vista Business

Hardware & Machines

Lapx

Hardware & Machines

My setup

Amplifier JVC MCA-V7

Hardware & Machines

Specs lapx

Elitebook 8570 ????

screen:

Screen 0: minimum 320 x 200, current 1600 x 900, maximum 16384 x 16384

LVDS connected primary 1600x900+0+0 (normal left inverted right x axis y axis) 345mm x 194mm

diameter: 15,8 inch

dpi: 116

SSD:

mtfddak256mam-1k12

Hardware & Machines

Tecra

BIOS: when keyboardlight comes on, press and hold F2

Hardware & Machines

Partitions lapx

tumbleweed:~ # fdisk -l /dev/sda
Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000c3032

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1            2048   4208639   4206592     2G 82 Linux swap / Solaris
/dev/sda2  *      4208640 109064893 104856254    50G 83 Linux
/dev/sda3       827269120 976773119 149504000  71.3G  f W95 Ext'd (LBA)
/dev/sda4       211466240 827269119 615802880 293.7G 83 Linux          /data       xfs
/dev/sda5       827271168 913856511  86585344  41.3G 83 Linux          / (tumbleweed, btrfs)
/dev/sda6       913858560 976752639  62894080    30G 83 Linux          /home (tumbleweed, xfs)

Partition table entries are not in disk order.

Hardware & Machines

Partitions tecra

partitions

sda1:         450mb      recovery      
sda2:         99mb
sda3:         100bytes erg kleinpartitietije
sda4:         50GB       C:
-----
sda5:         2GB        swap
sda6:         27GB       /
sda7:         40GB       /home

hdparm

tecra:~ # hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
        Model Number:       TOSHIBA THNSNK128GVN8                   
        Serial Number:      17CS14H7T8GT
        Firmware Revision:  K8TA4101
        Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Supported: 10 9 8 7 6 5 
        Likely used: 10
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  250069680
        LBA48  user addressable sectors:  250069680
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:      122104 MBytes
        device size with M = 1000*1000:      128035 MBytes (128 GB)
        cache/buffer size  = unknown
        Form Factor: unknown (0x0007]
        Nominal Media Rotation Rate: Solid State Device
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        Advanced power management level: 128
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4 
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
           *    Advanced Power Management feature set
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    WRITE_{DMA|MULTIPLE}_FUA_EXT
           *    64-bit World wide name
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Phy event counters
           *    READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
           *    DMA Setup Auto-Activate optimization
           *    Device-initiated interface power management
           *    Software settings preservation
                Device Sleep (DEVSLP)
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
           *    reserved 69[3]
           *    DOWNLOAD MICROCODE DMA command
           *    SET MAX SETPASSWORD/UNLOCK DMA commands
           *    WRITE BUFFER DMA command
           *    READ BUFFER DMA command
           *    DEVICE CONFIGURATION SET/IDENTIFY DMA commands
           *    Data Set Management TRIM supported (limit 8 blocks)
           *    Deterministic read ZEROs after TRIM
Security: 
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        2min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. 
Logical Unit WWN Device Identifier: 500080d9109a2088
        NAA             : 5
        IEEE OUI        : 00080d
        Unique ID       : 9109a2088
Device Sleep:
        DEVSLP Exit Timeout (DETO): 50 ms (drive)
        Minimum DEVSLP Assertion Time (MDAT): 10 ms (drive)
Checksum: correct

Hardware & Machines

System info tecra

tecra z50-c-14t

Hardware & Machines

Asus f2a85-v

CPU Chipset Geheugen Grafische Ondersteuning Multi-GPU Uitbreidingsslots Opslag LAN Audio AMD Socket FM2 Athlon™/A- Series Processors Supports CPU up to 4 cores Supports AMD® Turbo Core 3.0 Technology AMD A85X FCH(Hudson D4) 4 x DIMM, Max. 64GB, DDR3 2400(O.C.)/2250(O.C.)/2200(O.C.)/2133(O.C.)/2000(O.C.)/1866/1600/1333/1 066 MHz Non-ECC, Un-buffered Memory Dual Channel Memory Architecture

Refer to www.asus.com or this user manual for the Memory QVL (Qualified

Vendors Lists). Support AMD Memory Profile (AMP) memory

Due to OS limitation, when installing total memory of 4GB capacity or more,

Windows® 32-bit operation system may only recognize less than 3GB. Install a 64-bit WindowsWindows® OS when you want to install 4GB or more memory on the motherboard.

The Max. 64GB memory capacity can be supported with DIMMs of 16GB

(or above). ASUS will update QVL once the DIMMs are available on the market. Integrated AMD Radeon™ HD 7000 Series Graphics in Aseries APU Multi-VGA output support : HDMI/DVI/RGB ports - Supports HDMI with max. resolution 1920 x 1080 @ 60 Hz - Supports DVI with max. resolution 2560 x 1600 @ 60 Hz - Supports RGB with max. resolution 1920 x 1600 @ 60 Hz Maximum shared memory of 2048 MB AMD® Dual Graphics technology support * Supports DirectX 11 Supports AMD CrossFireX™ Technology 2 x PCIe 2.0 x16 (x16/x4) 2 x PCIe 2.0 x1 3 x PCI AMD A85X FCH(Hudson D4) chipset : 7 x SATA 6Gb/s ports (gray), gray 1 x SATA 6.0 Gb/s ports, red Support Raid 0, 1, 5, 10, JBOD Qualcomm Atheros, 1 x Gigabit LAN Controller Realtek® ALC887 8-Channel High Definition Audio CODEC - Supports : Jack-detection, Multi-streaming, Front Panel Jack-retasking Audio Feature : - Optical S/PDIF out port(s) at back panel

Hardware & Machines

Commodore64

video output info

Hardware & Machines

Hupsie

partitions-hupsie

Hardware & Machines

Asus P5Q Pro

[root]@amd64:~ # lspci
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:00.0 VGA compatible controller: NVIDIA Corporation G72 [GeForce 7500 LE] (rev a1)
02:00.0 Ethernet controller: Qualcomm Atheros AR8121/AR8113/AR8114 Gigabit or Fast Ethernet (rev b0)
[root]@amd64:~ #

Hardware & Machines

Hardware

20250404

# inxi -F
System:
  Host: amd64 Kernel: 6.14.0-lp156.4.g37149af-default arch: x86_64 bits: 64
    Console: pty pts/2 Distro: openSUSE Leap 15.6
Machine:
  Type: Desktop System: Gigabyte product: B550M K v: -CF serial: N/A
  Mobo: Gigabyte model: B550M K v: x.x serial: N/A BIOS: American
    Megatrends LLC. v: FD date: 03/22/2024
CPU:
  Info: 6-core model: AMD Ryzen 5 5600G with Radeon Graphics bits: 64
    type: MT MCP cache: L2: 3 MiB
  Speed (MHz): avg: 2506 min/max: 400/4465 cores: 1: 2367 2: 2367 3: 2367
    4: 2367 5: 2367 6: 2367 7: 2367 8: 3443 9: 2967 10: 2367 11: 2367 12: 2367
Graphics:
  Device-1: AMD Cezanne [Radeon Vega Series / Radeon Mobile Series]
    driver: amdgpu v: kernel
  Display: server: X.org v: 1.21.1.11 with: Xwayland v: 24.1.1 driver: X:
    loaded: amdgpu unloaded: fbdev,modesetting,vesa dri: radeonsi gpu: amdgpu
    resolution: 1920x1080~60Hz
  API: OpenGL v: 4.5 Mesa 23.3.4 renderer: llvmpipe (LLVM 17.0.6 256 bits)
Audio:
  Device-1: AMD Renoir Radeon High Definition Audio driver: snd_hda_intel
  API: ALSA v: k6.14.0-lp156.4.g37149af-default status: kernel-api
Network:
  Device-1: Realtek RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet
    driver: r8169
  IF: eth2 state: up speed: 1000 Mbps duplex: full mac: 10:ff:e0:7b:18:e2
RAID:
  Device-1: md1 type: mdraid level: raid-5 status: active size: 7.28 TiB
    report: 3/3 UUU
  Components: Online: 3: sdb 4: sdd1 5: sda
Drives:
  Local Storage: total: raw: 11.03 TiB usable: 7.39 TiB used: 4.95 TiB (67.0%)
  ID-1: /dev/sda vendor: Western Digital model: WD40EFAX-68JH4N0
    size: 3.64 TiB
  ID-2: /dev/sdb vendor: Western Digital model: WD40EFAX-68JH4N0
    size: 3.64 TiB
  ID-3: /dev/sdc vendor: TCSunBow model: X3 120GB size: 111.79 GiB
  ID-4: /dev/sdd vendor: Western Digital model: WD40EFAX-68JH4N0
    size: 3.64 TiB
Partition:
  ID-1: / size: 50 GiB used: 38.5 GiB (77.0%) fs: btrfs dev: /dev/sdc2
  ID-2: /home size: 54.98 GiB used: 27.08 GiB (49.2%) fs: btrfs
    dev: /dev/sdc4
  ID-3: /opt size: 50 GiB used: 38.5 GiB (77.0%) fs: btrfs dev: /dev/sdc2
  ID-4: /var size: 50 GiB used: 38.5 GiB (77.0%) fs: btrfs dev: /dev/sdc2
Swap:
  ID-1: swap-1 type: partition size: 6.8 GiB used: 0 KiB (0.0%) dev: /dev/sdc3
Sensors:
  System Temperatures: cpu: 28.8 C mobo: 27.0 C gpu: amdgpu temp: 24.0 C
  Fan Speeds (RPM): N/A
Info:
  Processes: 380 Uptime: 3h 58m Memory: available: 14.99 GiB
  used: 2.25 GiB (15.0%) Shell: Bash inxi: 3.3.27

20240714

amd64:~ # inxi -C
CPU:
  Info: 6-core model: AMD Ryzen 5 5600G with Radeon Graphics bits: 64
    type: MT MCP cache: L2: 3 MiB
  Speed (MHz): avg: 1046 min/max: 400/4464 cores: 1: 2989 2: 400 3: 400
    4: 400 5: 400 6: 400 7: 2974 8: 400 9: 400 10: 400 11: 400 12: 2994

amd64:~ # inxi -F
System:
  Host: amd64 Kernel: 6.9.7-lp155.2.ga5efffa-default arch: x86_64 bits: 64
    Console: pty pts/2 Distro: openSUSE Leap 15.6
Machine:
  Type: Desktop System: Micro-Star product: MS-7B89 v: 1.0 serial: N/A
  Mobo: Micro-Star model: B450M MORTAR MAX (MS-7B89) v: 1.0
    serial: K616819209 BIOS: American Megatrends LLC. v: 2.K0 date: 10/20/2023
CPU:
  Info: 6-core model: AMD Ryzen 5 5600G with Radeon Graphics bits: 64
    type: MT MCP cache: L2: 3 MiB
  Speed (MHz): avg: 616 min/max: 400/4464 cores: 1: 400 2: 400 3: 400 4: 400
    5: 400 6: 400 7: 400 8: 400 9: 400 10: 400 11: 2993 12: 400
Graphics:
  Device-1: AMD Cezanne [Radeon Vega Series / Radeon Mobile Series]
    driver: amdgpu v: kernel
  Display: server: X.org v: 1.21.1.11 with: Xwayland v: 23.2.4 driver: X:
    loaded: amdgpu unloaded: fbdev,modesetting,vesa dri: radeonsi gpu: amdgpu
    resolution: 1920x1080~60Hz
  API: OpenGL v: 4.5 Mesa 23.3.4 renderer: llvmpipe (LLVM 17.0.6 256 bits)
Audio:
  Device-1: AMD Renoir Radeon High Definition Audio driver: snd_hda_intel
  Device-2: Focusrite-Novation Focusrite Scarlett 2i2 2nd Gen
    driver: snd-usb-audio type: USB
  API: ALSA v: k6.9.7-lp155.2.ga5efffa-default status: kernel-api
Network:
  Device-1: Realtek RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet
    driver: r8169
  IF: eth1 state: up speed: 1000 Mbps duplex: full mac: 2c:f0:5d:5b:ce:91
RAID:
  Device-1: md1 type: mdraid level: raid-5 status: active size: 7.28 TiB
    report: 3/3 UUU
  Components: Online: 3: sdb 4: sda1 5: sdc
Drives:
  Local Storage: total: raw: 11.03 TiB usable: 7.39 TiB used: 4.96 TiB (67.1%)
  ID-1: /dev/sda vendor: Western Digital model: WD40EFAX-68JH4N0    size: 3.64 TiB
  ID-2: /dev/sdb vendor: Western Digital model: WD40EFAX-68JH4N0    size: 3.64 TiB
  ID-3: /dev/sdc vendor: Western Digital model: WD40EFAX-68JH4N0    size: 3.64 TiB
  ID-4: /dev/sdd vendor: TCSunBow model: X3 120GB                          size: 111.79 GiB
Partition:
  ID-1: / size: 50 GiB used: 35.58 GiB (71.2%) fs: btrfs dev: /dev/sdd2
  ID-2: /home size: 54.98 GiB used: 12.23 GiB (22.2%) fs: btrfs    dev: /dev/sdd4
  ID-3: /opt size: 50 GiB used: 35.58 GiB (71.2%) fs: btrfs dev: /dev/sdd2
  ID-4: /var size: 50 GiB used: 35.58 GiB (71.2%) fs: btrfs dev: /dev/sdd2
Swap:
  ID-1: swap-1 type: partition size: 6.8 GiB used: 0 KiB (0.0%) dev: /dev/sdd3
  ID-2: swap-2 type: file size: 9.54 GiB used: 0 KiB (0.0%)    file: /data/swap/pvdmswapfile
Sensors:
  System Temperatures: cpu: 29.0 C mobo: 33.0 C gpu: amdgpu temp: 27.0 C
  Fan Speeds (RPM): fan-1: 0 fan-2: 1197 fan-3: 0 fan-4: 0 fan-5: 0
Info:
  Processes: 402 Uptime: 0h 39m Memory: available: 13.54 GiB
  used: 4.16 GiB (30.7%) Shell: Bash inxi: 3.3.27

old:

amd64:~ # inxi -F
System:    Host: amd64 Kernel: 5.14.21-150400.24.21-default x86_64 bits: 64 Console: pty pts/0 Distro: openSUSE Leap 15.4
Machine:   Type: Desktop Mobo: Micro-Star model: B450M MORTAR MAX (MS-7B89) v: 1.0 serial: K616819209
           BIOS: American Megatrends LLC. v: 2.H0 date: 07/27/2022
CPU:       Info: 6-Core model: AMD Ryzen 5 5500 bits: 64 type: MT MCP cache: L2: 3 MiB
           Speed: 1413 MHz min/max: 1400/3600 MHz Core speeds (MHz): 1: 1413 2: 1359 3: 1421 4: 1397 5: 1397 6: 1397 7: 1397
           8: 1401 9: 1408 10: 1397 11: 1396 12: 1396
Graphics:  Device-1: NVIDIA GK208B [GeForce GT 730] driver: nvidia v: 470.141.03
           Display: server: SUSE LINUX 1.20.3 driver: loaded: nvidia unloaded: fbdev,modesetting,vesa
           resolution: 1920x1080~60Hz
           OpenGL: renderer: llvmpipe (LLVM 11.0.1 256 bits) v: 4.5 Mesa 21.2.4
Audio:     Device-1: NVIDIA GK208 HDMI/DP Audio driver: snd_hda_intel
           Device-2: Advanced Micro Devices [AMD/ATI] Renoir Radeon High Definition Audio driver: snd_hda_intel
           Sound Server-1: ALSA v: k5.14.21-150400.24.21-default running: yes
Network:   Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8168
           IF: eth1 state: up speed: 1000 Mbps duplex: full mac: 2c:f0:5d:5b:ce:91
RAID:      Device-1: md1 type: mdraid level: raid-5 status: active size: 7.28 TiB report: 3/3 UUU
           Components: Online: 3: sdc1 4: sdb1 5: sdd1
Drives:    Local Storage: total: raw: 11.03 TiB usable: 7.39 TiB used: 4.9 TiB (66.3%)
           ID-1: /dev/sda vendor: TCSunBow model: X3 120GB size: 111.79 GiB
           ID-2: /dev/sdb vendor: Western Digital model: WD40EFAX-68JH4N0 size: 3.64 TiB
           ID-3: /dev/sdc vendor: Western Digital model: WD40EFAX-68JH4N0 size: 3.64 TiB
           ID-4: /dev/sdd vendor: Western Digital model: WD40EFAX-68JH4N0 size: 3.64 TiB
Partition: ID-1: / size: 50 GiB used: 46.05 GiB (92.1%) fs: btrfs dev: /dev/sda2
           ID-2: /home size: 54.98 GiB used: 6.58 GiB (12.0%) fs: btrfs dev: /dev/sda4
           ID-3: /opt size: 50 GiB used: 46.05 GiB (92.1%) fs: btrfs dev: /dev/sda2
           ID-4: /tmp size: 50 GiB used: 46.05 GiB (92.1%) fs: btrfs dev: /dev/sda2
           ID-5: /var size: 50 GiB used: 46.05 GiB (92.1%) fs: btrfs dev: /dev/sda2
Swap:      ID-1: swap-1 type: partition size: 6.8 GiB used: 0 KiB (0.0%) dev: /dev/sda3
           ID-2: swap-2 type: file size: 9.54 GiB used: 0 KiB (0.0%) file: /data/swap/pvdmswapfile
Sensors:   System Temperatures: cpu: 26.0 C mobo: 31.0 C
           Fan Speeds (RPM): fan-1: 0 fan-2: 1566 fan-3: 0 fan-4: 0 fan-5: 0
Info:      Processes: 310 Uptime: 0h 44m Memory: 15.5 GiB used: 1.62 GiB (10.5%) Shell: Bash inxi: 3.3.07





OLD:=======================
Base Board Information
        Manufacturer: ASRock
        Product Name: FM2A88M Pro3+
        Version:
        Serial Number: M80-A3022300574


Processor Information
        Socket Designation: CPUSocket
        Type: Central Processor
        Family: A-Series
        Manufacturer: AMD
        ID: 31 0F 61 00 FF FB 8B 17
        Signature: Family 21, Model 19, Stepping 1
        Flags:
                FPU (Floating-point unit on-chip)
                VME (Virtual mode extension)
                DE (Debugging extension)
                PSE (Page size extension)
                TSC (Time stamp counter)
                MSR (Model specific registers)
                PAE (Physical address extension)
                MCE (Machine check exception)
                CX8 (CMPXCHG8 instruction supported)
                APIC (On-chip APIC hardware supported)
                SEP (Fast system call)
                MTRR (Memory type range registers)
                PGE (Page global enable)
                MCA (Machine check architecture)
                CMOV (Conditional move instruction supported)
                PAT (Page attribute table)
                PSE-36 (36-bit page size extension)
                CLFSH (CLFLUSH instruction supported)
                MMX (MMX technology supported)
                FXSR (FXSAVE and FXSTOR instructions supported)
                SSE (Streaming SIMD extensions)
                SSE2 (Streaming SIMD extensions 2)
                HTT (Multi-threading)
        Version: AMD A8-6600K APU with Radeon(tm) HD Graphics
        Voltage: 1.3 V
        External Clock: 100 MHz
        Max Speed: 3900 MHz
        Current Speed: 3900 MHz
        Status: Populated, Enabled
   Upgrade: Socket FM2
        L1 Cache Handle: 0x0008
        L2 Cache Handle: 0x0009
        L3 Cache Handle: Not Provided
        Serial Number: Not Specified
        Asset Tag: Not Specified
        Part Number: Not Specified
        Core Count: 4
        Core Enabled: 4
        Thread Count: 4
        Characteristics:
                64-bit capable

Hardware & Machines

Linux performance monitoring

Linux performance basics

I want to write about Cassandra performance tuning, but first I need to cover some basics: how to use vmstat, iostat, and top to understand what part of your system is the bottleneck -- not just for Cassandra but for any system.

vmstat

You will typically run vmstat with "vmstat sampling-period", e.g., "vmstat 5." The output looks like this:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
20  0 195540  32772   6952 576752   0    0    11    12   38   43  1  0 99  0
22  2 195536  35988   6680 575132   6    0  2952    14  959 16375 72 21 4  3

The first line is your total system average since boot; typically this will not be very useful, since you are interested in what is causing problems NOW. Then you will get one line per sample period; most of the output is self explanatory. The reason to start with vmstat is the "swap" section: si and so are swap in (memory read from disk) and swap out (memory written to disk). Remember that a little swapping is normal, particularly during application startup: by default, Linux will swap infrequently used pages of application memory to disk to free up more room for disk caching, even if there is enough ram to accommodate all applications.

iostat

To get more details of io, use iostat -x. Again, you want to give it a sampling interval, and ignore the first set of output. iostat also gives you some cpu information but top does that better; let's focus on the Device section:

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               9.80     0.20   36.60    0.40  5326.40     4.80   144.09     0.06    1.62   1.41   5.20

There are 3 easy ways to tell if a disk is a probable bottleneck here, and none of them show up without the -x flag, so get in the habit of using that. "avgqu-sz" is the size of the io request queue; if it is large, there are lots of requests waiting in line. "await" is how long (in ms) the average request took to be satisfied (including time enqueued); recall that on non-SSDs, a single seek is between 5 and 10ms. Finally, "%util" is Linux's guess at how fully saturated the device is.

top

To learn more about per-process CPU and memory usage, use "top." I won't paste top output here because everyone is so familiar with it, but I will mention a few useful things to know:

"P" and "M" toggle between sorting by cpu usage and sorting by memory usage
"1" toggles breaking down the CPU summary by CPU core
SHR (shared memory) is included in RES (resident memory)
Amount of memory belonging to a process that has been swapped out is VIRT - RES
a state (S column) of D means the process (or thread, see below) is waiting for disk or network i/o
"steal" is how much CPU the hypervisor is giving to another VM in a virtual environment; as virtual provisioning becomes more common, avoiding noisy neighbors is increasingly important

"top -H" will split out individual threads into their own lines; both per-process and per-thread views are useful. The per-thread view is particularly useful when dealing with Java applications since you can easily correlate them with thread names from the JVM to see which threads are consuming your CPU. Briefly, you take the PID (thread ID) from top, convert it to hex -- e.g., "python -c 'print hex(12345)'" -- and match it with the corresponding thread ID from jstack.

Now you can troubleshoot with a process like: "Am I swapping? If so, what processes are using all the memory? If my application makes a lot of disk read requests, are my reads being cached or are they actually hitting the disk? If I am hitting the disk, is it saturated? How much 'hot data' can I have before I run out of cache room? Are any/all of my cpu cores maxed? Which threads are actually using the CPU? Which threads spend most of their time waiting for i/o?" Then if you go to ask for help tuning something, you can show that you've done your homework.

Posted by Jonathan Ellis

Linux algemeen & CLI

Linux general

Linux algemeen & CLI

Image manipulation (The Gimp, DigiKam)

How to make a portion of an image transparent using The Gimp

Sometimes you want to make a certain portion of an image transparent.

This is for instance to make an image look good, placed on a background on a web page.

Remember that only .GIF and .PNG support this feature, NOT .JPG!! This can easily be done using The Gimp.

  1. Open your image in the Gimp. File -> Open.
  2. Right click the image and go to LAYERS then ADD ALPHA CHANNEL. This is the transparency layer.
  3. Right click on the image again and go to SELECT and then BY COLOR.
  4. Now click on the color in the image you want to be transparent. These colors will now show up outlined.
  5. Right click on the image again and go to EDIT and then down to CLEAR.
     This should now erase the outlined color you just picked from the image and the "transparent gimp checkerbox" should show through.
     This is the Gimps way of showing you that section is now transparent.
  6. Right click on the image and choose SAVE AS and make sure to save as a GIF file if you want the transparency to work on the web.
     Select CONVERT TO INDEXED if necessary. It is also possible to SAVE AS... PNG
  7. That's it!

Linux algemeen & CLI

Linux and hardware

Linux algemeen & CLI

Watch

Herrie http://www.herrie.info commandline mp3 player/jukebox
Thunder & Lightning 3D war simulator http://tnlgame.net
WeedIt File duplicate scanner http://adm1n.cjb.net/cw
Bacula http://www.bacula.org
MyRPM Create RPMs without fuss http://code.google.com/p/myrpm
RockBox Open source firmware replacement for many popular MP3 players
http://bluemarine.tidalwave.it/ An open source application for the digital photo workflow
http://www.linux.com/articles/57222 tesseract en other OCR meuk
http://groundstate.ca/ocr ocropus
http://www.openfiler.com opensource NAS
http://www.andlinux.org/ run linux from within wondows
http://www.ossec.net/ OSSEC is an Open Source Host-based Intrusion Detection System. It performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response
http://www.jokosher.org/ jokosher audio editor
Zenmap graphical interface for nmap
slowloris apache killler
Unetbootin install any os on usb stick
http://pianoteq.com piano synthesizer
http://luma.sourceforge.net -> graphical LDAP browser
http://www.php-web-statistik.de/index-english.html php webstat
http://download.bitdefender.com/SMB/Workstation_Security_and_Management/BitDefender_Antivirus_Scanner_for_Unices/Unix/Current/EN_FR_BR_RO/Linux/ bitdefender antivirus scanner
http://kde-apps.org/content/show.php/Hydrogen?content=14152 Hydrogen KDE Sound Application
http://www.jolicloud.com jolicloud, netbook OS
http://www.openvas.org openvas, fork from nessus
http://www.roomle.com roomle, online woninginrichting
gbrainy brain trainer
typolight lightweight CMS

Linux algemeen & CLI

How to find duplicate files BY CONTENT!!

Today in IRC suseROCKS needed to find all duplicate files in a directory by their content, not by their file name, so we whipped up this fancy little 1 liner bash script to do the trick:

find . -type f -exec md5sum '{}' \; | sort | awk 'dup[$1]++{print $2}'

EDIT:

As Andreas suggested, using xargs instead of -exec is much faster, here is the updated command:

find . -type f -print0 | xargs -0 md5sum | sort | awk 'dup[$1]++{print $2}'

Linux algemeen & CLI

How to clone or copy your harddisk over the network

Method 1 (tested!)

On the bron/zender/source:

#dd if=/dev/sda | gzip -c | netcat -l -q 0 -p 2222

On the ontvanger/doel/target:

#netcat 2222 | gzip -cd | dd of=/dev/sda

This way we get a throughput of 14,1 MB/s, 160 GB in 11543 seconds (3,2 hour)

Methode 2 (niet getest!)

How do I use netcat to copy hard disk image?

Our sample setup

HostA // 192.168.1.1
          sda
       NETWORK
          sdb
HostB // 192.168.1.2

Your task is copy HostA /dev/sda to HostB's /dev/sdb using netcat command. First login as root user Command to type on hostB (receiving end ~ write image mode)

You need to open port on hostB using netcat, enter :

# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb

Where,

   * -p 2222 : Specifies the source port nc should use, subject to privilege restrictions and availability.   Make sure port 2222 is not used by another process.
   * -l : Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.
   * bzip2 -d : Compresses image using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. This will speed up network transfer ( -d : force decompression mode)
   * dd of=/dev/sda : /dev/sda is your hard disk. You can also specify partition such as /dev/sda1

Command to type on hostA (send data over a network ~ read image mode)

Now all you have to do is start copying image. Again login as root and enter:

# bzip2 -c /dev/sda | netcat hostA 2222

OR use IP address:

# bzip2 -c /dev/sda | netcat 192.168.1.1 2222

This process takes its own time. A note about latest netcat version 1.84-10 and above

If you are using latest nc / netcat version above syntax will generate an error. It is an error to use -l option in conjunction with the -p, -s, or -z options. Additionally, any timeouts specified with the -w option are ignored. So use nc command as follows.

On hostA, enter:

# nc -l 2222 > /dev/sdb

On hostB, enter:

# nc hostA 2222< /dev/sda

# nc 192.168.1.1 2222< /dev/sda

Using a second machine (hostB), connect to the listening nc process at 2222 (hostA), feeding it the file (/dev/sda)which is to be transferred. You can use bzip2 as follows. On hostA, enter:

# nc -l 2222 | bzip2 -d > /dev/sdb

On hostB, enter:

# bzip2 -c /dev/sda | nc 192.168.1.1 2222

Linux algemeen & CLI

How to find information about a host or webserver

nmap -sV -P0 -O [server]

Linux algemeen & CLI

Linux commandline tips

Bash tips

^r      Reverse search the command history
^a  Return to the start of the command you're typing
^e  Go to the end of the command you're typing
^u  Cut everything before the cursor to a special clipboard
^k  Cut everything after the cursor to a special clipboard
^y  Paste from the special clipboard that Ctrl + u and Ctrl + k save their data to
^t  Swap the two characters before the cursor (you can actually use this to transport a character from the left to the right, try it!)
^w  Delete the word / argument left of the cursor
^l  Clear the screen

history    lists command history
!61        executes command on line 61

ls -l /etc/passwd
stat !$             no need to repeat argument on second line

man -k <keyword>     searches in all man pages for <keyword>

ls *.mp3 | xargs rm
ls *.mp3 | xargs -i cp {} /home/bla
ls *.mp3 -d  -> stays in current dir and doesn't enter mp3/

start a restricted shell (no cd, etc) to test programs

# bash -r

start shell with options

# bash -O [modification]
autocd -> use the argument as cd command
cd-spell -> corrects dir names
dirspell -> corrects dir names in file completion
no-caseglob -> case insensitive

to run commands:

# command1 && command2

logical AND :: run command1, if ok then run command2

# command1 || command2

logical OR :: run command2 only if command1 fails

cp

cp -rv

gives feedback

cp -p

preserves original date/time

dd

use blocks of 64k and report every 10Mb:

dd if=/dev/hda | buffer -s 64k -S 10m > image

or, from other console:

kill -SIGUSR1 $(pidof dd)

df

df -type=ext3,ReiserFS
df -H human readable

find

find all files in your homedir modified or created today

find ~ -type f -mtime 0

case insentive search

find . -iname "*mp3"

find files and directories in yor homedir not created by you

find ~ ! -user ${USER}

and fix this

find ~ ! -user ${USER} -exec sudo chown ${USER}:"{}" \;

find duplicates PvdM

find . -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 15

free

free -m -> megabytes
free -t -> totals
free -s[seconds] -> update interval

watch the line with +/- buffers: free indicates memory that can be freed instantaniously used is all memory being used

grep

grep -A2 bla greps 2 lines after
grep -B3 bla greps 3 lines before

history

# history -d [postition] -> remove specific command
# history -c -> clear entire history

iotop

monitors io proces

iotop -o makes output more readable by showing only processes causing io right now
iotop -d10 refresh interval
iotop -o -b -d10 -n30 > io.txt batch mode: write io status 30 times to file at 10s interval

iperf

iperf -s -w128k run server for performance benchmark
iperf -c -w128k -t30 -r run client
iperf -c -w128k -t30 -d run client in two directions, full duplex

kill

kill -l shows instructions

1  SIGHUP   restart immediately after terminating, or triggers reconfiguration of background service
2  SIGINT   Ctrl+c from the keyboard, terminate!
3  SIGQUIT  Ctrl+\ from the keyboard, terminate with coredump
9  SIGKILL  force terminate, extreme signal that can't ignored
11  SIGSEGV  program attempted an invalid memory reference, terminate with coredump
15  SIGTERM  request to terminate and cleanup
19  SIGSTOP  interrupts the process until you enter SIGCONT to continue

kill -19 9102 10234  stops several processes
killall -19 ssh      sends all ssh connections to sleep
killall -19 -i ssh   interactive mode (recommended)

lsof

lsof list open files
lsof -i :22 list all ssh connections (same :ssh)
lsof -i@10.0.0.1 list all connections from 10.0.0.1
lsof -i@amsterdam.nl list all connections from domain
lsof -u username list all open files from username
lsof -c bash list open files related to bash
lsof -c /log/ list all open files from all processes containing log

lspci

lspci -v
lspci -vv
lspci -vvv
lspci -nn -> show names & numerical
lspci -k -> show kernel modules associated with the device

more or less

+ number of line from which to start displaying
z number of lines to jump forward
y number of lines to jump back
-I ignore case when searching
-V underline tabs, line endings
/ highlights all ocurrences of search

pgrep

pgrep ssh lists all ssh processes
pgrep -l ssh shows names
pgrep -f shows full commandline
pgrep -u show user

ping

ping 0 -> pings localhost
ping c 5 nu.nl -> pings 5 times
ping -f host -> floods the host
ping -a IP -> gives audible beep Note: It can give beep only from terminal number 1 through 7 and gnome-terminal ( It will not work in console ).
ping -q ip -> show only summary
ping -s 100 -> change packet size
Pressing CTRL+| (Control key followed by pipe symbol) for the shows the summary in between, and continues with it packet sending and receiving process.
ping hop1 hop2 hop3 .. hopN destination
ping -R 192.168.1.63 -> Record and print route of how ECHO_REQUEST sent and ECHO_REPLY received

pkill

understands the same options as pgrep

pkill -19 ssh
pkill -19 -u joop stop joop's processes

ps

show how long a process is running

ps -o pid,etime,cmd `pidof amarok`

show memory usage per process

ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS

pstree

pstree -a shows processes tree structure with program's parameters
pstree -h highlights own process
pstree -H highlights process id
pstree -u shows user

stat

shows status of file

stat /etc/passwd
stat -c "%x %n" /etc/passwd formatted to show time

swapon

swapon -s see size and usage

tail or head

--retry keep monitoring the file
-s seconds how often monitoring retry
-pid stops tail from running if PID stops running

tar

tar zxf -> extracts file

top

PID     programma id
USER    gebruiker
PR      prioriteit van de taak
NI      nice value
VIRT    virtuele geheugen gebruik
RES     fysieke geheugen gebruik
SHR     deel v/h gebeugen dat mogelijk met andere taken wordt gedeeld
%CPU    percentage processorgebruik
%MEM    percentage geheugengebruik
TIME+   de tijd dat de taak op de cpu heeft gedraaid
COMMAND de naam van de taak

keys: z: enable color mode. x: show sorted column in different color < >: change sorted column u [user]: show only users processes d: delay in sec

show only user user

#top -u user

zombies

ps aux | grep defunct

AutoYast

create autoyast file through icon in Yast2->autoyast

www.suse.de/~ug (Uwe Gansert)

Midnight Commander (MC)

The standard colorpallette is incredibly hard to read in certain terminals, like yakuake.

For readable colors (mostly black and green), enter in your ~/.mc/ini file the following line:

[Colors]
base_color=lightgray,green:normal=green,default:selected=white,gray:marked=yellow,default:markselect=yellow,gray:directory=blue,default:executable=brightgreen,default:link=cyan,default:device=brightmagenta,default:special=lightgray,default:errors=red,default:reverse=green,default:gauge=green,default:input=white,gray:dnormal=green,gray:dfocus=brightgreen,gray:dhotnormal=cyan,gray:dhotfocus=brightcyan,gray:menu=green,default:menuhot=cyan,default:menusel=green,gray:menuhotsel=cyan,default:helpnormal=cyan,default:editnormal=green,default:editbold=blue,default:editmarked=gray,blue:stalelink=red,default

tune kernelparameters

stored in /proc/sys/ gone after reboot

sysctl is a service that reads sysctl.conf during boot and read parameters from there -> makes it permanent

# ulimit -a

givesinfo about system resources limitations

Linux algemeen & CLI

Using x2x to connect two computers and their screens with one mouse and keyboard

Let's say you have 2 computers:

cpu1: 10.0.0.1
cpu2: 10.0.0.2

Let's say cpu1's screen is left to cpu2's screen.

Let's say you want to control cpu1 from cpu2.

~~Let's assume both computers have the same username.~~

install x2x on cpu1
enter on cpu2

ssh -X user@cpu1 'x2x -west -to :0'

and leave terminal open.

And that's it!

Linux algemeen & CLI

Script to check and restart a process


#!/bin/bash
# -------------------------------------------------------------------------
# Copyright (c) 2003 nixCraft project <http://cyberciti.biz/fb/>
# This script is licensed under GNU GPL version 2.0 or above
# -------------------------------------------------------------------------
# This script is part of nixCraft shell script http://en.wikipedia.org/wiki/Shell_script collection (NSSC)
# Visit http://bash.cyberciti.biz/ for more information.
# Copyright (c) 2010 CyberOrg Info. Modification to watch any process/service 
# -------------------------------------------------------------------------
RESTART="/etc/init.d/sshd restart"
PGREP="/usr/bin/pgrep"
processtowatch="sshd"
x=1
while [ $x = 1 ];
do 
    # find pid
    $PGREP ${processtowatch}
    if [ $? -ne 0 ]
    then
        echo "`date` : $processtowatch went down, restarting" >> /var/log/restart.log
        $RESTART
    fi
    sleep 30
done

Linux algemeen & CLI

Vi tips

vi editor tips

Keys

Key(s):	Movement:
0 (zero)	1^st column on the line
^	1^st non-blancchar on line
$	Last char on line
g_	Last non-blanc char on line
w	Next word
W	Next word and skip interpunction
e	End of current word
E	End of current word incl .interpunction
b	Begin of current word
B	Begin of current word incl. interpuction
ge	End of previous word
gE	End of previous word and skip interpunction
O	Insert new line and switch to insert mode

Key(s):	Action:
x	Delete char under cursor
X	Delete char before cursor
r	Replace char under cursor with next typed char

d	Delete chars until position of next command
dd	Delete line
y	Copy text until position of next command
yy	Copy line
p	Paste text aftercursor
P	Paste text beforecursor

*Key(s):*	*Selection:*
V	Start selecting block of text
x	Cut block of text
c	Copy block of text
iw	Word
aw	Word incl spaces
is	Sentence
as	Sentence incl spaces
ip	Paragraph
ap	Paragraph incl spaces and empty lines
i(	Block between brackets
a(	Block incl brackets
i<	Block between < >
a<	Block incl < >
i{	Block between { }
a{	Block incl { }
i”	Block between “ “
a”	Block incl“ “
i`	Block between ` `
a`	Block incl` `

*Search:*	*Action:*
/text	Search for text starting at cursor
n	Find next
N	Find previous
//	Repeat last search
*	Search forward for word currently under cursor
#	Search backwards for word currently under cursor

:set hlsearch	Highlights all matches
:set nohlsearch	Disable ^
:nohl	Don't show highlights from last search
:set incsearch	Enables incremental search
:set noincsearch	Disables ^

u undo

cut multiple lines: ###yy

delete rest of line from cursor: SHIFT-d

To open multiple files on a window

Horizontal split

vi -o file-one file-two

Vertical split

vi -O file-one file-two

Switch between the panes you are using.

     CTRL + W + Left key to activate left windows
     CTRL + W + Right key to activate right windows
     CTRL + W + Up key> to activate to windows above current one
     CTRL + W + Down key> to activate to windows down current one

Split Open one file at a time

vi file-one

To open the second file, go to the command mode (Esc)

:new file-two

:split file-two

to split vertically

:vsplit

Typ 'CTRL-w w' to switch between the windows
Open new file:

:e newfile

Tabs

To open Vi with multiple tabs open:

vi -p /var/log/*.log

switch between tabs with gt
open new tab

:tabnew
or
:tabe file

to open new file in new tab

Search and Replace

Go in command mode and type:

Replace First occurence of string

:%s/string-to-replace/replace-with-string/

Replace All occurence of string

:%s/string-to-replace/replace-with-string/g

Replace All occurence of string with confirmation

:%s/string-to-replace/replace-with-string/gc

avoid staircase effect when pasting

If you're working in Vim and paste something into the terminal, sometimes you'll get a "staircase" effect where each line is progressively spaced farther outward, like so:

line 1
 line 2
  line 3
   line 4

Obviously, this isn't usually desirable.

To correct this, you can enable paste to prevent the staircase effect. In command mode, type:

:set paste

This isn't on by default. When paste is enabled, it disables mapping and some other functions, so you probably want to know how to turn it off as well:

:set nopaste

Highlighting

syntax highlighting for programming languages

:syntax on

Store settings

Put your most used commands, like :syntax on in ~/.vimrc

statusline

set statusline=%F%m%r%h%w%=\ [%Y]\ [%{&ff}]\ [%04l,%04v]\ [%p%%]\ [%L]

in /etc/vimrc

Linux algemeen & CLI

Verhuis een linux systeem naar nieuwe hardware

Verhuis van systeem A naar systeem B.

Inpakken

boot systeem A vanaf liveCD

: (alternatief: boot in single-user-mode)

mount de rootpartitie van systeem A:

mount /dev/sda1 /mnt

mount het medium waarop het archief komt. Vaak automatisch, anders:

mount /dev/sdf1 /media

ga naar root van (gemounte) filesysteem

cd /mnt

(als je gebruikt maakt van de single-user-mode, dan cd / uiteraard)
pak in:

tar --one-file-system -cpf /media/systeem.tar * (-j=compressie)

Uitpakken

boot systeem B vanaf liveCD
maak nieuwe disk aan, en partitioneer. Bijvoorbeeld:

fdisk /dev/sda

: denk er ook aan om een swap partitie aan te maken (type 82)

maak bestandssystemen aan:

mkfs.ext3 /dev/sda1
mkswap /dev/sda2
etc....

mount de doelpartitie en de gegevensdrager:

mount /dev/sda1 /mnt
mount /dev/sdf1 /media

pak het systeem uit:

cd /mnt
tar -xf /media/systeem.tar (-j als je compressie gebruikt)

Inrichten

pas /etc/fstab aan, vervang bijvoorbeeld UUID's door apparaatnamen zoals /dev/sda1

/dev/sda1 / ext3 acl,user_xattr 1 1
/dev/sda2 swap swap defaults 0 0

verwijder 2 udev regels voor netwerkkaarten en optische drives:

rm /mnt/etc/udev/rules.d/70-persistent-cd.rules
rm /mnt/etc/udev/rules.d/70-persistent-net.rules

: deze worden na booten opnieuw aangemaakt.

pas /mnt/etc/X11/xorg.conf aan of wis deze zodat deze opnieuw wordt aangemaakt.

pas GRUB aan:
- pas de 'root=' regel aan
- vervang in /mnt/boot/grub/menu.lst UUID's door apparaatnamen zoals /dev/sda1

mount nu de proc en dev bestandssystemen:

mount -t proc none /mnt/proc
mount -o bind /dev /mnt/dev

ga dan met chroot naar het nieuwe systeem:

chroot /mnt

en installeer grub bootloader:

grub-install /dev/sda (exit met CRTL-D)

Linux algemeen & CLI

A guide to using systemd

This is how to stop a running service temporarily:

systemctl stop servicename.service

Send a kill:

systemctl kill sshd.service
systemctl kill -s HUP sshd.service

This stops it from starting at boot, but does not stop a running service:

systemctl disable servicename.service

And there is one way to really really stop a service for good, short of uninstalling it, and that is masking it by linking it to /dev/null:

ln -s /dev/null /etc/systemd/system/servicename.service
systemctl daemon-reload

When you do this you can't even start the service manually. Nothing can touch it.

What if you change your mind? Pish tosh, it's easy. Simply delete the symlink and run

systemctl enable servicename.service.

While we're here, let's talk about two reload commands: daemon-reload and reload. The daemon-reload option reloads the entire systemd manager configuration without disrupting active services. reload reloads the configuration files for specific services without disrupting service, like this:

systemctl reload servicename.service

This reloads the actual configuration file used by the hardy sysadmin, for example the /etc/ssh/sshd_config file for an SSH server, and not its systemd unit file, sshd.service. So this is what to use when you make configuration changes.

List active service units:

systemctl --type=service

list all service units:

systemctl --type=service --all

list systemd services:

systemctl list-unit-files

default target

The system boots to default.target

This is a symbolic link to multiuser.target or graphical.target.

Changing the default runlevel/target is replacing the symbolic link

ln -sf /lib/systemd/system/multi-user.target /etc/systemd/system/default.target

To change runlevel:

systemctl isolate multi-user.target

booting

At boot time, on the kernel grub line, add:

systemd.unit=xxxxx

So what's taking so long? We can find out with the systemd-analyze blame command

$ systemd-analyze blame
  60057ms sendmail.service
  51241ms firstboot-graphical.service
  3574ms sshd-keygen.service
  3439ms NetworkManager.service
  3101ms udev-settle.service
  3025ms netfs.service
  2411ms iptables.service
  2411ms ip6tables.service
  2173ms abrtd.service
  2149ms nfs-idmap.service
  2116ms systemd-logind.service
  2097ms avahi-daemon.service
  1337ms iscsi.service

If you like pretty graphs systemd includes a cool command for automatically generating an SVG image from the blame output, like this:

systemd-analyze plot > graph1.svg

To shutdown:

systemctl poweroff

systemctl files

/lib/systemd/system/
/etc/systemd/system/ is for customized settings

Linux algemeen & CLI

Perform bechmarks on the filesystem

To test the speed of the filesystem (not the physical disk!) you can use the following script:


#!/bin/bash

# USAGE:
# ./speed_test.sh /path/to/my/file /path/to/destination number_of_tests

NUM_TESTs=$3
SUM=0
for i in $( seq 1 $NUM_TESTs ); do

REC=`dd if=$1 of=$2 2> some_random_file_ ; cat some_random_file_ | cut -d " " -f8 | tail -1`

SUM=`echo $SUM + $REC | bc`

done

RESULT=`echo $SUM / $NUM_TESTs | bc |  awk '{ str1=str1 $0 }END{ print str1 }'`

echo $RESULT MB/s

#clean up
rm some_random_file_
rm $2

To make a big file, use

cat /dev/zero > zerofile

and abort it with CTRL-C after a few seconds.

Alternatively, you can use something like

dd if=/dev/zero of=file.out bs=1MB count=500

to create a 500MB file.

Start the script like this:

./speed_test.sh /mnt/sda1/zerofile /mnt/sdb1/zerofilecopy 3

The result will be something like

./speed_test.sh ./zerofile ./2delete 5
237 MB/s

Linux algemeen & CLI

How to backup your disk with dd

I had several bad experiences losing my data in my life. Sometimes because of the hardware, sometimes because of my stupidness or inexperience. At least, it learned to be careful. I backup my data regularly, and on several support concerning the important ones.

I sometimes back up my primary partition, where the system is set. If my drive crashed, I want to recover quickly a system without having to reinstall hundreds of applications. I have used several times Partimage and was satisfied. But now I tend to use dd, which basically do the same thing in just one line. The advantage is that it is by default on any Linux distribution.

To back it up on an usb disk :

$ sudo dd if=/dev/hda1 | gzip -v | dd of=/media/usbdisk/backup_hda1.gz

To backup my home and other data partitions, I just copy the files manually, it is faster and there is no need to backup such a thing as the boot sector. But of course you could use the same command above.

When you wish to restore the partition to a new hard drive, just :

$ zcat /mnt/hdb5/sauvegarde.gz | dd of=/dev/hda1

To do that, you will have probably booted on a live CD Linux as Knoppix to restore the crashed system. In any case, don’t overwrite the system you just booted on !

One tip if you want to back up only the boot sector :

$ sudo dd if=/dev/hda of=/home/secteur_boot.dd bs=512 count=1

And to restore :

$ sudo dd if=/home/secteur_boot.dd of=/dev/hda bs=512 count=1

Advanced options

oflag=direct,sync

for writing to slow devices such as SD cards, oflag=direct keeps it from filling up the process's memory with the backlogged output, and oflag=sync keeps it from filling up the kernel's memory.

direct is a good idea if you want to avoid buffering on the write end, but sync will severely degrade performance if the drive you're writing to does its own buffering. It will force every write to flush the drive's buffer too before moving on.

also: status=progress

Linux algemeen & CLI

Things to clean up before doing large copy of entire system

FF cache
chromium cache
.thumbnails
spotify localstorecache
.wine
/var/tmp/kdecache-pvdm

Linux algemeen & CLI

Linux commandline tips 4

trap ctrl-c in terminal:

trap 'play -nq synth sin 1000 trim 0 0.35 vol 0.5 2>/dev/null' INT

watch if a website changes:

while :;do curl -Ls "X\.com"|md5sum;sleep 5m;done|awk '{if(NR>1&&l!=$1){system("echo the site changed|mail -s NOTIFY you@isp\.net");};l=$1}'

scan pdf to file. first extract the pages and ocr them, then make one doc

pdfimages -tiff input.pdf plaatje
for i in *.tif; do tesseract $i tempje-$i; done
cat tempje-plaatje-0*.txt >> docje.txt

ps ax -o state -o ppid | awk '$1=="Z"{print $2}' | xargs kill -9 # Kill all #zombies on the system.

ps aux |tail -n+2 |sort -nrk4 |head -$(($(tput lines)-1)) |cut -c 1-$(tput cols) # Display top RAM using processes.

ethtool -p eth0 # Blink eth0's LED so you can find it in the rat's next of server cables. Ctrl-C to stop.

find . -xdev -ls | sort -n -k 7 | tail -5 # Quickly find the largest 5 files in the CWD tree without crossing filesystem boundaries.

for f in *; do b=$(echo "$f" | tr '[A-Z]' '[a-z]'); mv "$f" "$b"; done # Lower case all files in a folder.

rpm -qf $( which lspci ) # Pass the output of which (showing path to lspci) into rpm's -qf, which tells you the pkg.

rename -v 's/^([0-9])_/0\1_/' *.flac # Rename all single leading digit flac files so that they have a padding 0 for easier sorting.

ps aux | awk '{if ($8=="Z") { print $2 }}'# On Linux, print out a list of the process IDs that are in the zombie state.

exiv2 -k -F rename *.jpg # Use the exiv2 EXIF program to rename your jpg files according to their exif date/time data.

curl http://wttr.in/castricum # see weatherforecast

finger amsterdam@graph.no

rpm -qa --queryformat "%{NAME} %{INSTALLTIME:date}\n" | grep "Nov 2015" # In RPM, determine which packages where installed in Nov 2015.

echo "wall \"hallo\""| at 11:48 2016-02-27 # execute a command at a certain time and date

ls -la "$(find . -type f -printf '%T@ %p\n' | sort -n | tail -1 | cut -f2- -d" ")" # show the most recent file in subtree

grep -Ev "((accept|drop|reject)log|ftpd)" /var/log/messages | less # Yes! You can do nested grouping in extended regexes.

With Bash history, `^str1^str2^` will repeat the previous command replacing `str1` by `str2`, so

echo 1223
^2^4^

The #Bash has the notion of "integer" variable:

declare A=1; declare -i B=2
A+=3; B+=4
echo $A $B
⇒ 13 6

sort -t, -k5nr data.csv | less # Sort data.csv by the 5th column's numeric values in descending order.

Play a sound if you get an unsuccessful return code from last command.

PROMPT_COMMAND='[ $? -ne 0 ] && play -qn synth sin G3 trim 0 0.1'

Command used by P2V process:

tar --one-file-system --sparse -C / -cf - .

Show error warn and criticals in color:

# tail -f "foo.log"|egrep --line-buffered --color=auto 'ERROR|WARN|CRITICAL$

Very good overview of open ports and their programs:

# lsof -Pan -i tcp -i udp

To remove the hashtag parameter in a file and add the line if not exist; thus making sure the parameter is always present and enabled:

# grep -qi 'ForwardToSyslog' /etc/systemd/journald.conf && sed -i 's/#ForwardToSyslog.*/ForwardToSyslog=Yes/' /etc/systemd/journald.conf || echo 'ForwardToSyslog=Yes' >> /etc/systemd/journald.conf

Linux algemeen & CLI

Linux algemeen

fuser -ck /net

pid of proccess holding mount

openssl x509 -noout -in scx-host-sldbowzs0101.basis-a.acc.pem -text

def quickSort(arr):
    less = []
    pivotList = []
    more = []
    if len(arr) <= 1:
        return arr
    else:

def quickSort(arr):
    less = []
    pivotList = []
    more = []
    if len(arr) <= 1:
        return arr
    else:

Linux algemeen & CLI

Awk

awk

pattern {action}
pattern {action}

$ awk '{print $2}' /etc/fstab

to filter out comment lines - starting with #

$ awk '/^[^#]/ {print $2}' /etc/fstab

to grep on TWO occurences (AND operator)

$ awk '/pattern1/ && /pattern2/' /var/log/messages

case insensitive

$ awk '/pattern1/ && /pattern2/' IGNORECASE=1 /var/log/messages

print first field from password file using field seperator ':'

$ awk -F: '{print$1}' /etc/passwd

print only user ID > 500

$ awk -F: '$3>=500 {print$1}' /etc/passwd

display average of numbers entereed on a line:

$ awk '{ sum=0; for (i=1; i<=NF; i++) sum +=$i; print sum/NF; }'

NF	number of fields in the current line
NR	the current record number (line number)
FS	input field seperator (space by default)
RS	record seperator (newline by default)
Example	Example
Example	Example
Example	Example
Example	Example
Example	Example

show the 5th record on the second line:

df /tmp | awk 'NR==2 {print$5 }'

use awk with a program

the file maxuid:

BEGIN {maxuid = 0; FS=":" }
{ if ($3 > maxuid) maxuid = $3 }
END { print" the largest UID is ", maxuid }

then use awk like this:

$ awk -f maxuid /etc/passwd

to ignore user nobody:

BEGIN {maxuid = 0; FS=":" }
$1 != "nobody" { if ($3 > maxuid) maxuid = $3 }
END { print" the largest UID is ", maxuid }

cat -n: (adds line numbers)

$ awk '{ print NR, $0}' /etc/fstab

wc -w: (count words)

$ awk '{ w += NF } END { print w}' /etc/fstab

grep chris /etc/passwd:

$ awk '/^chris/' /etc/passwd

writing self contained scripts

#!/usr/bin/awk -f
{ cost[$1] += $2*$4 }
END { for (cat in cost) print cat, cost[cat] }

chmod u+x catscript
./catscript

http://www.pement.org/awk/awk1line.txt

Linux algemeen & CLI

Systemd journal log file setting

The new openSUSE 13.2 journald (from systemd) is sometimes quickly using up disk space.

There are setting you can tweak in /etc/systemd/journald.conf

[Journal]
Compress=yes
SystemMaxUse=500M
SystemKeepFree=2G

After that, restart the services a few times for the changes to take effect:

systemctl restart systemd-journald.service
systemctl restart systemd-journal-flush.service

You should see a reduction in the size of the /var/log/ directory almost immediately.

Linux algemeen & CLI

Dealing with spaces in filenames

BASH for loop works nicely under UNIX / Linux / Windows and OS X while working on set of files. However, if you try to process a for loop on file name with spaces in them you are going to have some problem. For loop uses $IFS variable to determine what the field separators are. By default $IFS is set to the space character. There are multiple solutions to this problem.

Set $IFS variable

Try it as follows:

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
for f in *
do
  echo "$f"
done
IFS=$SAVEIFS

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
# set me
FILES=/data/*
for f in $FILES
do
  echo "$f"
done
# restore $IFS
IFS=$SAVEIFS

More examples using $IFS and while loop

Now you know that if the field delimiters are not whitespace, you can set IFS. For example, while loop can be used to get all fields from /etc/passwd file:

....
while IFS=: read userName passWord userID groupID geCos homeDir userShell
do
      echo "$userName -> $homeDir"
done < /etc/passwd

source: http://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html

Linux algemeen & CLI

View progress of dd

In Linux, find running dd processes and send them a signal to print out their progress.

kill -USR1 $( pidof dd )

Linux algemeen & CLI

Sort installed rpms by size

Sort installed RPMs by size

To list all RPM packages installed on a Linux machine sorted from the largest to the smallest in size:

rpm -qa --queryformat '%{size} %{name}\n' | sort -rn | more

Linux algemeen & CLI

Command line tips

Show all VMs:

vboxmanage list -l vms

Show the state of the VM:

vboxmanage showvminfo w7new | grep State

Control the state of the VM:

Linux algemeen & CLI

Find

find latest files in all subdirectories per directory

Linux algemeen & CLI

Find latest files in all subdirectories per directory

Create a file latestfile.sh:

 #!/bin/bash
 ls -lrt "$1" | tail -n1

Make it executable

chmod +x ./latestfile.sh

Then execute the following command:

find . -mindepth 1 -type d -printf "%T+ %p/" -exec ./latestfile.sh {} \;

Output:

2015-02-07+04:01:53.0000000000 ./$RCE/SLES11-SP2-Updates/sle-11-i586/repodata/repomd.xml.key
2015-02-07+04:02:09.0000000000 ./$RCE/SLES11-SP2-Updates/sle-11-i586/repodata
2015-02-07+04:07:41.0000000000 ./$RCE/SLES11-SP3-Pool/sle-11-i586/.repodata/repomd.xml.key
2015-02-07+04:07:41.0000000000 ./$RCE/SLES11-SP3-Pool/sle-11-i586/rpm

When outputted to a file (list.txt) then you could follow up with:

cat list.txt | grep -v repomd | grep rpm | sort

Linux algemeen & CLI

Journald

output since booting:

journalctl -b

Follow:

journalctl -f

Keep an eye on sshd service:

journalctl -b -f -l -u sshd

Examples:

journalctl -b --system
journalctl -b --user

journalctl --since=yesterday

journalclt -b _PID=1
journalclt -b _UID=1001
journalclt -b /usr/bin/sudo
journalctl KERNEL_DEVICE=+scsi:0:0:0:0

-x -> extra info

To monitor the output of dmesg:

# journalctl -kf

# dmesg -wH

Linux algemeen & CLI

Sample commands

testdatabase 'testsietske';

> mysql -u root

MariaDB [testsietske]> use testsietske;

MariaDB [testsietske]> show tables;

MariaDB [testsietske]> select * from City;

MariaDB [testsietske]> select * from City where Countrycode = "NLD";

MariaDB [testsietske]> select * from City where District = "Noord-Holland";

MariaDB [testsietske]> select * from City where Countrycode = "NLD" and  population > "100000";


MariaDB [testsietske]> show tables;

MariaDB [testsietske]> select * from Country where Continent = "Europe";

----

You can use db_name.tbl_name as an alternative to the tbl_name FROM db_name syntax. In other words, these two statements are equivalent:

 SHOW COLUMNS FROM City;
 MariaDB [testsietske]> SHOW COLUMNS FROM mytable FROM mydb;
 MariaDB [testsietske]> SHOW COLUMNS FROM mydb.mytable;

show only first 10 results

 MariaDB [testsietske]> select * from City limit 10;

show result 10 to 30

 MariaDB [testsietske]> select * from City limit 20 offset 10;

Linux algemeen & CLI

Bash fun

pong:

yes $COLUMNS $LINES|awk 'BEGIN{x=y=e=f=1}{if(x==$1||!x){e*=-1};if(y==$2||!y){f*=-1};x+=e;y+=f;printf "\033[%s;%sH",y,x;system("sleep .02")}'

Linux algemeen & CLI

OCR

scan pdf to file. first extract the pages and ocr them, then make one doc

pdfimages -tiff input.pdf plaatje
for i in *.tif; do tesseract $i tempje-$i; done
cat tempje-plaatje-0*.txt >> docje.txt

Use tesseract to OCR a multi-page PDF file

First, convert the PDF to multiple TIFF files, because tesseract does not work with PDF.

Number the files with 2 digits at the end, remove the alfa-channel:

# convert -density 300 inputfile.pdf -depth 8 -alpha off outputfile_%02d.tiff

Then, use tesseract to make it into text:

# tesseract inputfile.tiff outputfile

if you do not provide and extension for the outputfile, it will become .txt

Creating an overlay with the OCRed text

The newer version of Tesseract (3.03 RC at the time of writing this) can do this:

free, opensource and cross-platform
starting from version 3.03 PDF output is available
CLI software
multiple languages support
unfortunately, single image input, so to make a complete document, one must create a batch script to convert each page image to searchable PDF. After that PDF pages should be combined to a single PDF using tools like pdftk.

This is the command:

tesseract -l input.tif output pdf

Note that in order to use this approach, the input PDF has to be rasterized first, since tesseract will not get PDF as input.

To combine multiple PDF files into one

# pdfunite output_*.pdf result.pdf

I have created a script

This script will do the work for you. Place the script in a directory together with the PDF to be processed, and run it.

#!/bin/bash

# converts a PDF containing scanned pages into a 
# new PDF file in which the OCRed text is overlayed,
# making the PDF searchable on text strings.

# use: 
#       doit.sh nice.PDF

# requires: tesseract-ocr, convert (ImageMagick), pdfunite, pdfinfo

# 20170417      1.0     PvdM    first version
# 20170418      1.1     PvdM    minor adjustment and improvements, mainly in the counter

bestand="$1"
newbestand=$(echo $bestand | cut -d"." -f1)_searchable.pdf
teller="0"
teller2="000"
aantpaginas=$(pdfinfo "$bestand" | grep 'Pages:' | awk '{ print $2 }')
RESTORE='\033[0m'
RED='\033[00;31m'
GREEN='\033[00;32m'
YELLOW='\033[00;33m'
BLUE='\033[00;34m'
PURPLE='\033[00;35m'
CYAN='\033[00;36m'
LIGHTGRAY='\033[00;37m'

function check_input {
if [ -z "$bestand" ]; then
        echo - Error. Usage:
        echo "         ./doit.sh input.pdf"; echo
        exit 1
fi
}

function check_error {
        if [ $? != 0 ]; then
                echo == Error! There was a problem in the command.
                exit 1
        fi
}

clear
echo "Converting PDF to searchable (overlay) PDF."
echo -e "-------------------------------------------\n"
check_input
echo -e " $bestand contains $RED $aantpaginas $RESTORE pages.\n"
echo " - (1/3) Extracting scanned PDF to images........"
convert -density 300 "$bestand" -depth 8 -alpha off temp_%03d.tiff
check_error
echo -e " - Done.\n"
echo

echo " - (2/3) Doing OCR on the images..........."
for i in temp_*.tiff; do
        tesseract -l eng $i temp_pdf_$teller2.pdf pdf
        check_error
        ((teller++))
        teller2=$(printf "%05d" $teller)
        echo -e " - (2/3) Doing OCR on the images. $RED Page $teller/$aantpaginas done.$RESTORE" 
done
echo -e " - Done.\n"
echo

echo " - (3/3) Combining the result into 1 (searchable) PDF"
pdfunite temp_pdf_*.pdf "$newbestand"
check_error
echo -e " - Done. $RED'$newbestand'$RESTORE created.\n"
rm temp*

examples

how to extract images from pdf

pdfimages -all sm_td20a_very_detailed.pdf .

pdfimages -f 40 -l 41 -tiff hfe_teac_x-300_300r_service.pdf power-pcb

Linux algemeen & CLI

How to get into bios-setup

BIOS: when keyboardlight comes on, press and hold F2

Linux algemeen & CLI

Ace

@tecra:~> acestreamengine --client-console

@tecra:~> acestream-launcher -p vlc acestream://67665adae9b7535a10f7b7a22d43f15683e78bde

https://acestreamid.com/channel/sky-sports-f1

Linux algemeen & CLI

.vimrc

To support the creation and editing of yaml files in vi, set the following in ~/.vimrc

set expandtab
set tabstop=2
set autoindent
set shiftwidth=2
set softtabstop=2
set colorcolumn=3,5,7,9,11
set number

Linux algemeen & CLI

Tips and tricks

Probably the best way to do this - assuming that you can't use the NOPASSWD solution provided by scottod is to use Mircea Vutcovici's solution in combination with Ansible vault.

For example, you might have a playbook something like this:

- hosts: all

  vars_files:
    - secret

  tasks:
    - name: Do something as sudo
      service: name=nginx state=restarted
      sudo: yes

Here we are including a file called secret which will contain our sudo password.

We will use ansible-vault to create an encrypted version of this file:

ansible-vault create secret

This will ask you for a password, then open your default editor to edit the file. You can put your ansible_sudo_pass in here.

e.g.: secret:

ansible_sudo_pass: mysudopassword

Save and exit, now you have an encrypted secret file which Ansible is able to decrypt when you run your playbook. Note: you can edit the file with ansible-vault edit secret (and enter the password that you used when creating the file)

The final piece of the puzzle is to provide Ansible with a --vault-password-file which it will use to decrypt your secret file.

Create a file called vault.txt and in that put the password that you used when creating your secret file. The password should be a string stored as a single line in the file.

From the Ansible Docs:

.. ensure permissions on the file are such that no one else can access your key and do not add your key to source control

Finally: you can now run your playbook with something like

ansible-playbook playbook.yml -u someuser -i hosts --sudo --vault-password-file=vault.txt

The above is assuming the following directory layout:

.
|_ playbook.yml
|_ secret
|_ hosts
|_ vault.txt

You can read more about Ansible Vault here: https://docs.ansible.com/playbooks_vault.html

Linux algemeen & CLI

Performance harddisk

orginele harddisk

/dev/sda:

 Model=TOSHIBA THNSNK128GVN8, FwRev=K8TA4101, SerialNo=17CS14H7T8GT
 Config={ Fixed }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=250069680
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: Unspecified:  ATA/ATAPI-3,4,5,6,7

 * signifies the current active mode

tecra:~ # hdparm -T /dev/sda

/dev/sda:
 Timing cached reads:   13074 MB in  1.99 seconds = 6579.23 MB/sec
tecra:~ # hdparm -T /dev/sda

/dev/sda:
 Timing cached reads:   13082 MB in  1.99 seconds = 6583.21 MB/sec
^[[Atecra:~ # hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 1428 MB in  3.00 seconds = 475.35 MB/sec
tecra:~ # hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 1404 MB in  3.00 seconds = 467.92 MB/sec
tecra:~ #

evo 970

Linux algemeen & CLI

Bash pitfalls

Bash Pitfalls

This page shows common errors that Bash programmers make. These examples are all flawed in some way.

You will save yourself from many of these pitfalls if you simply always use quotes and never use WordSplitting for any reason! Word splitting is a broken legacy misfeature inherited from the Bourne shell that's stuck on by default if you don't quote expansions. The vast majority of pitfalls are in some way related to unquoted expansions, and the ensuing word splitting and globbing that result.

<<Anchor(pf1)>>

for f in $(ls *.mp3)

One of the most common mistakes BASH programmers make is to write a loop like this:

for f in $(ls *.mp3); do    # Wrong!
    some command $i         # Wrong!
done

for f in $(ls)              # Wrong!
for f in `ls`               # Wrong!

for f in $(find . -type f)  # Wrong!
for f in `find . -type f`   # Wrong!

files=($(find . -type f))   # Wrong!
for f in ${files[@]}        # Wrong!

Yes, it would be great if you could just treat the output of `ls` or `find` as a list of filenames and iterate over it. But you cannot. This entire approach is fatally flawed, and there is no trick that can make it work. You must use an entirely different approach.

There are at least 6 problems with this:

If a filename contains whitespace, it undergoes WordSplitting. Assuming we have a file named `01 - Don't Eat the Yellow Snow.mp3` in the current directory, the `for` loop will iterate over each word in the resulting file name: 01, -, Don't, Eat, etc.
If a filename contains glob characters, it undergoes filename expansion ("globbing"). If `ls` produces any output containing a * character, the word containing it will become recognized as a pattern and substituted with a list of all filenames that match it.
If the command substitution returns multiple filenames, there is no way to tell where the first one ends and the second one begins. Pathnames may contain any character except NUL. Yes, this includes newlines.
The `ls` utility may mangle filenames. Depending on which platform you're on, which arguments you used (or didn't use), and whether its standard output is pointing to a terminal or not, `ls` may randomly decide to replace certain characters in a filename with "?", or simply not print them at all. Never try to parse the output of ls. `ls` is just plain unnecessary. It's an external command whose output is intended specifically to be read by a human, not parsed by a script.
The CommandSubstitution strips all trailing newline characters from its output. That may seem desirable since `ls` adds a newline, but if the last filename in the list ends with a newline,
```
`...`
```
or `$()` will remove that one also.
In the `ls` examples, if the first filename starts with a hyphen, it may lead to pitfall #3.

You can't simply double-quote the substitution either:

for f in "$(ls *.mp3)"; do     # Wrong!

This causes the entire output of `ls` to be treated as a single word. Instead of iterating over each file name, the loop will only execute once, assigning to `f` a string with all the filenames rammed together.

Nor can you simply change IFS to a newline. Filenames can also contain newlines.

Another variation on this theme is abusing word splitting and a `for` loop to (incorrectly) read lines of a file. For example:

IFS=$'\n'
for line in $(cat file); do ...     # Wrong!

This doesn't work! Especially if those lines are filenames. Bash (or any other Bourne family shell) just doesn't work this way.

So, what's the right way to do it?

There are several ways, primarily depending on whether you need a recursive expansion or not.

If you don't need recursion, you can use a simple glob. Instead of `ls`:

for file in ./*.mp3; do    # Better! and...
    some command "$file"   # ...always double-quote expansions!
done

POSIX shells such as Bash have the globbing feature specifically for this purpose -- to allow the shell to expand patterns into a list of matching filenames. There is no need to interpret the results of an external utility. Because globbing is the very last expansion step, each match of the `./*.mp3` pattern correctly expands to a separate word, and isn't subject to the effects of an unquoted expansion.

Question: What happens if there are no *.mp3-files in the current directory? Then the for loop is executed once, with i="./*.mp3", which is not the expected behavior! The workaround is to test whether there is a matching file:

# POSIX
for file in ./*.mp3; do
    [ -e "$file" ] || continue
    some command "$file"
done

Another solution is to use Bash's `shopt -s nullglob` feature, though this should only be done after reading the documentation and carefully considering the effect of this setting on all other globs in the script.

If you need recursion, the standard solution is `find`. When using find, be sure you use it properly. For POSIX sh portability, use the `-exec` option:

find . -type f -name '*.mp3' -exec some command {} \;

# Or, if the command accepts multiple input filenames:

find . -type f -name '*.mp3' -exec some command {} +

If you're using bash, then you have two additional options. One is to use GNU or BSD `find`'s `-print0` option, together with bash's `read -d ''` option and a ProcessSubstitution:

while IFS= read -r -d '' file; do
  some command "$file"
done < <(find . -type f -name '*.mp3' -print0)

The advantage here is that "some command" (indeed, the entire `while` loop body) is executed in the current shell. You can set variables and have them persist after the loop ends.

The other option, available in Bash 4.0 and higher, is `globstar`, which permits a glob to be expanded recursively:

shopt -s globstar
for file in ./**/*.mp3; do
  some command "$file"
done

Note the double quotes around `$file` in the examples above. This leads to our second pitfall:

<<Anchor(pf2)>>

cp $file $target

What's wrong with the command shown above? Well, nothing, if you happen to know in advance that `$file` and `$target` have no white space or wildcards in them. However, the results of the expansions are still subject to WordSplitting and pathname expansion. Always double-quote parameter expansions.

cp -- "$file" "$target"

Without the double quotes, you'll get a command like `cp 01 - Don't Eat the Yellow Snow.mp3 /mnt/usb`, which will result in errors like

cp: cannot stat `01': No such file or directory

. If `$file` has wildcards in it (* or ? or [), they will be expanded if there are files that match them. With the double quotes, all's well, unless "$file" happens to start with a `-`, in which case `cp` thinks you're trying to feed it command line options (See pitfall #3 below.)

Even in the somewhat uncommon circumstance that you can guarantee the variable contents, it is conventional and good practice to quote parameter expansions, especially if they contain file names. Experienced script writers will always use quotes except perhaps for a small number of cases in which it is absolutely obvious from the immediate code context that a parameter contains a guaranteed safe value. Experts will most likely consider the `cp` command in the title always wrong. You should too.

<<Anchor(pf3)>>

Filenames with leading dashes

Filenames with leading dashes can cause many problems. Globs like `*.mp3` are sorted into an expanded list (according to your current locale), and `-` sorts before letters in most locales. The list is then passed to some command, which may incorrectly interpret the `-filename` as an option. There are two major solutions to this.

One solution is to insert `--` between the command (like `cp`) and its arguments. That tells it to stop scanning for options, and all is well:

cp -- "$file" "$target"

There are potential problems with this approach. You have to be sure to insert `--` for every usage of the parameter in a context where it might possibly be interpreted as an option -- which is easy to miss and may involve a lot of redundancy.

Most well-written option parsing libraries understand this, and the programs that use them correctly should inherit that feature for free. However, still be aware that it is ultimately up to the application to recognize end of options. Some programs that manually parse options, or do it incorrectly, or use poor 3rd-party libraries may not recognize it. Standard utilities should, with a few exceptions that are specified by POSIX. `echo` is one example.

Another option is to ensure that your filenames always begin with a directory by using relative or absolute pathnames.

for i in ./*.mp3; do
    cp "$i" /target
    ...
done

In this case, even if we have a file whose name begins with `-`, the glob will ensure that the variable always contains something like `./-foo.mp3`, which is perfectly safe as far as `cp` is concerned.

Finally, if you can guarantee that all results will have the same prefix, and are only using the variable a few times within a loop body, you can simply concatenate the prefix with the expansion. This gives a theoretical savings in generating and storing a few extra characters for each word.

for i in *.mp3; do
    cp "./$i" /target
    ...
done

<<Anchor(pf4)>>

== [ $foo = "bar" ] == This is very similar to the issue in pitfall #2, but I repeat it because it's so important. In the example above, the quotes are in the wrong place. You do not need to quote a string literal in bash (unless it contains metacharacters or pattern characters). But you should quote your variables if you aren't sure whether they could contain white space or wildcards.

This example can break for several reasons:

* If a variable referenced in `[` doesn't exist, or is blank, then the `[` command would end up looking like:
.

[ = "bar" ] # Wrong!

. ...and will throw the error: `unary operator expected`. (The `=` operator is binary, not unary, so the `[` command is rather shocked to see it there.)

* If the variable contains internal whitespace, then it gets split into separate words before the `[` command sees it. Thus:
.

[ multiple words here = "bar" ]

. While that may look OK to you, it's a syntax error as far as `[` is concerned. The correct way to write this is:
.

# POSIX
[ "$foo" = bar ] # Right!

. This works fine on POSIX-conformant implementations even if `$foo` begins with a `-`, because POSIX `[` determines its action depending on the number of arguments passed to it. Only very ancient shells have a problem with this, and you shouldn't worry about them when writing new code (see the `x"$foo"` workaround below).

In Bash and many other ksh-like shells, there is a superior alternative which uses the [[BashFAQ/031|keyword.

# Bash / Ksh
[[ $foo == bar ]] # Right!

You don't need to quote variable references on the left-hand side of `=` in `` because they don't undergo word splitting or globbing, and even blank variables will be handled correctly. On the other hand, quoting them won't hurt anything either. Unlike `[` and `test`, you may also use the identical `==`. Do note however that comparisons using `` perform pattern matching against the string on the right hand side, not just a plain string comparison. To make the string on the right literal, you must quote it if any characters that have special meaning in pattern matching contexts are used. <pre> # Bash / Ksh match=b*r [[ $foo == "$match" # Good! Unquoted would also match against the pattern b*r.

You may have seen code like this:

# POSIX / Bourne
[ x"$foo" = xbar ] # Ok, but usually unnecessary.

The `x"$foo"` hack is required for code that must run on very ancient shells which lack [[BashFAQ/031|, and have a more primitive `[`, which gets confused if `$foo` begins with a `-`. On said older systems, `[` still doesn't care whether the token on the right hand side of the `=` begins with a `-`. It just uses it literally. It's just the left-hand side that needs extra caution.

Note that shells that require this workaround are not POSIX-conforming. Even the Heirloom Bourne shell doesn't require this (probably the non-POSIX Bourne shell clone that's still most widely in use as a system shell). Such extreme portability is rarely a requirement and makes your code less readable (and uglier).

<<Anchor(pf5)>>

cd $(dirname "$f")

This is yet another quoting error. As with a variable expansion, the result of a CommandSubstitution undergoes WordSplitting and pathname expansion. So you should quote it:

cd -P -- "$(dirname -- "$f")"

What's not obvious here is how the quotes nest. A C programmer reading this would expect the first and second double-quotes to be grouped together; and then the third and fourth. But that's not the case in Bash. Bash treats the double-quotes inside the command substitution as one pair, and the double-quotes outside the substitution as another pair.

Another way of writing this: the parser treats the command substitution as a "nesting level", and the quotes inside it are separate from the quotes outside it.

<<Anchor(pf6)>>

== [ "$foo" = bar && "$bar" = foo ] == You can't use `&&` inside the old test (or [) command. The Bash parser sees `&&` outside of `` or `(( ))` and breaks your command into two commands, before and after the `&&`. Use one of these instead:

[ bar = "$foo" ] && [ foo = "$bar" ] # Right! (POSIX)
[[ $foo = bar && $bar = foo ]]       # Also right! (Bash / Ksh)

(Note that we reversed the constant and the variable inside `[` for the legacy reasons discussed in pitfall #4. We could also have reversed the `[[` case, but the expansions would require quoting to prevent interpretation as a pattern.) The same thing applies to `||`. Either use `[[` instead, or use two `[` commands.

Avoid this:

[ bar = "$foo" -a foo = "$bar" ] # Not portable.

The binary `-a` and `-o`, and `(` / `)` (grouping) operators are XSI extensions to the POSIX standard. All are marked as obsolescent in POSIX-2008. They should not be used in new code. One of the practical problems with `[ A = B -a C = D ]` (or `-o`) is that POSIX does not specify the results of a `test` or `[` command with more than 4 arguments. It probably works in most shells, but you can't count on it. If you have to write for POSIX shells, then you should use two `test` or `[` commands separated by a `&&` operator instead.

<<Anchor(pf7)>>

$foo > 7

There are multiple issues here. First, the [[BashFAQ/031|command should not be used solely for evaluating arithmetic expressions. It should be used for test expressions involving one of the supported test operators. Though technically you can do math using some of ``'s operators, it only makes sense to do so in conjunction with one of the non-math test operators somewhere in the expression. If you just want to do a numeric comparison (or any other shell arithmetic), it is much better to just use `(( ))` instead: <pre> # Bash / Ksh ((foo > 7)) # Right! [[ foo -gt 7 # Works, but is pointless. Most will consider it wrong. Use ((...)) or let instead.

If you use the `>` operator inside ``, it's treated as a string comparison (test for collation order by locale), not an integer comparison. This may work sometimes, but it will fail when you least expect it. If you use `>` inside `[ ]`, it's even worse: it's an output redirection. You'll get a file named `7` in your directory, and the test will succeed as long as `$foo` is not empty.

If strict POSIX-conformance is a requirement, and `((` is not available, then the correct alternative using old-style `[` is

# POSIX
[ "$foo" -gt 7 ]       # Also right!
[ $((foo > 7)) -ne 0 ] # POSIX-compatible equivalent to ((, for more general math operations.

Note that the `test ... -gt` command will fail in interesting ways if `$foo` is not an integer. Therefore, there's not much point in quoting it properly other than for performance and to confine the arguments to a single word to reduce the likelihood of obscure side-effects possible in some shells.

If the input to any arithmetic context (including `((` or `let`), or `[` test expression involving numeric comparisons can't be guaranteed then you must always validate your input before evaluating the expression.

# POSIX
case $foo in
    *[![:digit:]]*)
        printf '$foo expanded to a non-digit: %s\n' "$foo" >&2
        exit 1
        ;;
    *)
        [ $foo -gt 7 ]
esac

<<Anchor(pf8)>>

grep foo bar | while read -r; do ((count++)); done

The code above looks OK at first glance, doesn't it? Sure, it's just a poor implementation of `grep -c`, but it's intended as a simplistic example. Changes to `count` won't propagate outside the `while` loop because each command in a pipeline is executed in a separate SubShell. This surprises almost every Bash beginner at some point.

POSIX doesn't specify whether or not the last element of a pipeline is evaluated in a subshell. Some shells such as ksh93 and Bash >= 4.2 with `shopt -s lastpipe` enabled will run the `while` loop in this example in the original shell process, allowing any side-effects within to take effect. Therefore, portable scripts must be written in such a way as to not depend upon either behavior.

For workarounds for this and similar issues, please see Bash FAQ #24. It's a bit too long to fit here.

<<Anchor(pf9)>>

if [grep foo myfile]

Many beginners have an incorrect intuition about `if` statements brought about by seeing the very common pattern of an `if` keyword followed immediately by a `[` or `[[`. This convinces people that the `[` is somehow part of the `if` statement's syntax, just like parentheses used in C's `if` statement.

This is not the case! `if` takes a command. `[` is a command, not a syntax marker for the `if` statement. It's equivalent to the `test` command, except that the final argument must be a `]`. For example:

# POSIX
if [ false ]; then echo "HELP"; fi
if test false; then echo "HELP"; fi

are equivalent -- both checking that the argument "false" is non-empty. In both cases HELP will always be printed, to the surprise of programmers from other languages guessing about shell syntax.

The syntax of an `if` statement is:

if COMMANDS
then <COMMANDS>
elif <COMMANDS> # optional
then <COMMANDS>
else <COMMANDS> # optional
fi # required

Once again, `[` is a command. It takes arguments like any other regular simple command. `if` is a compound command which contains other commands -- and there is no [ in its syntax!

While bash has a builtin command `[` and thus `knows` about `[` it has nothing special to do with `]`. Bash only passes `]` as argument to the `[` command, which requires `]` to be the last argument only to make scripts look better.

There may be zero or more optional `elif` sections, and one optional `else` section.

The `if` compound command is made up of two or more sections containing lists of commands, each delimited by a `then`, `elif`, or `else` keyword, and is terminated by the `fi` keyword. The exit status of the final command of the first section and each subsequent `elif` section determines whether each corresponding `then` section is evaluated. Another `elif` is evaluated until one of the `then` sections is executed. If no `then` section is evaluated, then the `else` branch is taken, or if no `else` is given, the `if` block is complete and the overall `if` command returns 0 (true).

If you want to make a decision based on the output of a `grep` command, you do not want to enclose it in parentheses, brackets, backticks, or any other syntax! Just use `grep` as the `COMMANDS` after the `if`, like this:

if grep -q fooregex myfile; then
...
fi

If the `grep` matches a line from `myfile`, then the exit code will be 0 (true), and the `then` part will be executed. Otherwise, if there are no matches, `grep` will return non-zero and the overall `if` command will be zero.

See also:

* BashGuide/TestsAndConditionals
* http://wiki.bash-hackers.org/syntax/ccmd/if_clause

<<Anchor(pf10)>>

== if [bar="$foo"]; then ... ==

[bar="$foo"]   # Wrong!
[ bar="$foo" ] # Still wrong!

As explained in the previous example, `[` is a command (which can be proven with `type -t [` or `whence -v [`). Just like with any other simple command, Bash expects the command to be followed by a space, then the first argument, then another space, etc. You can't just run things all together without putting the spaces in! Here is the correct way:

if [ bar = "$foo" ]; then ...

Each of `bar`, `=`, the expansion of `"$foo"`, and `]` is a separate argument to the `[` command. There must be whitespace between each pair of arguments, so the shell knows where each argument begins and ends.

<<Anchor(pf11)>>

== if [ [ a = b ] && [ c = d ] ]; then ... == Here we go again. `[` is a command. It is not a syntactic marker that sits between `if` and some sort of C-like "condition". Nor is it used for grouping. You cannot take C-like `if` commands and translate them into Bash commands just by replacing parentheses with square brackets!

If you want to express a compound conditional, do this:

if [ a = b ] && [ c = d ]; then ...

Note that here we have two commands after the `if`, joined by an `&&` (logical AND, shortcut evaluation) operator. It's precisely the same as:

if test a = b && test c = d; then ...

If the first `test` command returns false, the body of the `if` statement is not entered. If it returns true, then the second `test` command is run; and if that also one returns true, then the body of the `if` statement will be entered. (C programmers are already familiar with `&&`. Bash uses the same short-circuit evaluation. Likewise `||` does short-circuit evaluation for the OR operation.)

The [[BashFAQ/031|keyword does permit the use of `&&`, so it could also be written this way:

if [[ a = b && c = d ]]; then ...

See pitfall #6 for a pitfall related to tests combined with conditional operators.

<<Anchor(pf12)>>

read $foo

You don't use a `$` before the variable name in a `read` command. If you want to put data into the variable named `foo`, you do it like this:

.

 read foo

Or more safely:

.

 IFS= read -r foo

`read $foo` would read a line of input and put it in the variable(s) whose name(s) are in `$foo`. This might be useful if you actually intended `foo` to be a reference to some other variable; but in the majority of cases, this is simply a bug.

<<Anchor(pf13)>>

cat file | sed s/foo/bar/ > file

You cannot read from a file and write to it in the same pipeline. Depending on what your pipeline does, the file may be clobbered (to 0 bytes, or possibly to a number of bytes equal to the size of your operating system's pipeline buffer), or it may grow until it fills the available disk space, or reaches your operating system's file size limitation, or your quota, etc.

If you want to make a change to a file safely, other than appending to the end of it, use a text editor.

.

 printf %s\\n ',s/foo/bar/g' w q | ed -s file

If you are doing something that cannot be done with a text editor there must be a temporary file created at some point(*). For example, the following is completely portable:

.

 sed 's/foo/bar/g' file > tmpfile && mv tmpfile file

The following will only work on GNU sed 4.x:

.

 sed -i 's/foo/bar/g' file(s)

Note that this also creates a temporary file, and does the same sort of renaming trickery -- it just handles it transparently.

And the following equivalent command requires perl 5.x (which is probably more widely available than GNU sed 4.x):

.

 perl -pi -e 's/foo/bar/g' file(s)

For more details on replacing contents of files, please see Bash FAQ #21.

(*) `sponge` from moreutils uses this example in its manual:

.

 sed '...' file | grep '...' | sponge file

Rather than using a temporary file plus an atomic `mv`, this version "soaks up" (the actual description in the manual!) all the data, before opening and writing to the `file`. This version will cause data loss if the program or system crashes during the write operation, because there's no copy of the original file on disk at that point.

Using a temporary file + `mv` still incurs a slight risk of data loss in case of a system crash / power loss; to be 100% certain that either the old or the new file will survive a power loss, you must use `sync` before the `mv`.

<<Anchor(pf14)>>

echo $foo

This relatively innocent-looking command causes massive confusion. Because the `$foo` isn't quoted, it will not only be subject to WordSplitting, but also file globbing. This misleads Bash programmers into thinking their variables contain the wrong values, when in fact the variables are OK -- it's just the word splitting or filename expansion that's messing up their view of what's happening.

.

 msg="Please enter a file name of the form *.zip"
 echo $msg

This message is split into words and any globs are expanded, such as the *.zip. What will your users think when they see this message:

.

 Please enter a file name of the form freenfss.zip lw35nfss.zip

To demonstrate:

.

 var="*.zip"   # var contains an asterisk, a period, and the word "zip"
 echo "$var"   # writes *.zip
 echo $var     # writes the list of files which end with .zip

In fact, the `echo` command cannot be used with absolute safety here. If the variable contains `-n` for example, `echo` will consider that an option, rather than data to be printed. The only absolutely sure way to print the value of a variable is using `printf`:

.

 printf "%s\n" "$foo"

<<Anchor(pf15)>>

== $foo=bar == No, you don't assign a variable by putting a `$` in front of the variable name. This isn't perl.

<<Anchor(pf16)>>

== foo = bar == No, you can't put spaces around the `=` when assigning to a variable. This isn't C. When you write `foo = bar` the shell splits it into three words. The first word, `foo`, is taken as the command name. The second and third become the arguments to that command.

Likewise, the following are also wrong:

.

 foo= bar    # WRONG!
 foo =bar    # WRONG!
 $foo = bar; # COMPLETELY WRONG!

 foo=bar     # Right.
 foo="bar"   # More Right.

<<Anchor(pf17)>>

echo <<EOF

A here document is a useful tool for embedding large blocks of textual data in a script. It causes a redirection of the lines of text in the script to the standard input of a command. Unfortunately, `echo` is not a command which reads from stdin.

.

  # This is wrong:
  echo <<EOF
  Hello world
  How's it going?
  EOF

  # This is what you were trying to do:
  cat <<EOF
  Hello world
  How's it going?
  EOF

  # Or, use quotes which can span multiple lines (efficient, echo is built-in):
  echo "Hello world
  How's it going?"

Using quotes like that is fine -- it works great, in all shells -- but it doesn't let you just drop a block of lines into the script. There's syntactic markup on the first and last line. If you want to have your lines untouched by shell syntax, and don't want to spawn a `cat` command, here's another alternative:

.

  # Or use printf (also efficient, printf is built-in):
  printf %s "\
  Hello world
  How's it going?
  "

In the `printf` example, the `\` on the first line prevents an extra newline at the beginning of the text block. There's a literal newline at the end (because the final quote is on a new line). The lack of `\n` in the printf format argument prevents `printf` adding an extra newline at the end. The `\` trick won't work in single quotes. If you need/want single quotes around the block of text, you have two choices, both of which necessitate shell syntax "contaminating" your data:

.

  printf %s \
  'Hello world
  '

  printf %s 'Hello world
  '

<<Anchor(pf18)>>

su -c 'some command'

This syntax is almost correct. The problem is, on many platforms, `su` takes a `-c` argument, but it's not the one you want. For example, on OpenBSD:

.

 $ su -c 'echo hello'
 su: only the superuser may specify a login class

You want to pass `-c 'some command'` to a shell, which means you need a username before the `-c`.

.

 su root -c 'some command' # Now it's right.

`su` assumes a username of root when you omit one, but this falls on its face when you want to pass a command to the shell afterward. You must supply the username in this case.

<<Anchor(pf19)>>

cd /foo; bar

If you don't check for errors from the `cd` command, you might end up executing `bar` in the wrong place. This could be a major disaster, if for example `bar` happens to be `rm -f *`.

You must always check for errors from a `cd` command. The simplest way to do that is:

.

 cd /foo && bar

If there's more than just one command after the `cd`, you might prefer this:

.

 cd /foo || exit 1
 bar
 baz
 bat ... # Lots of commands.

`cd` will report the failure to change directories, with a stderr message such as "bash: cd: /foo: No such file or directory". If you want to add your own message in stdout, however, you could use command grouping:

.

 cd /net || { echo >&2 "Can't read /net. Make sure you've logged in to the Samba network, and try again."; exit 1; }
 do_stuff
 more_stuff

Note there's a required space between `{` and `echo`, and a required `;` before the closing `}`. You could also write a `die` function, if you prefer.

Some people also like to enable set -e to make their scripts abort on any command that returns non-zero, but this can be rather tricky to use correctly (since many common commands may return a non-zero for a warning condition, which you may not want to treat as fatal).

By the way, if you're changing directories a lot in a Bash script, be sure to read the Bash help on `pushd`, `popd`, and `dirs`. Perhaps all that code you wrote to manage `cd`'s and `pwd`'s is completely unnecessary.

Speaking of which, compare this:

.

 find ... -type d -print0 | while IFS= read -r -d '' subdir; do
   here=$PWD
   cd "$subdir" && whatever && ...
   cd "$here"
 done

With this:

.

 find ... -type d -print0 | while IFS= read -r -d '' subdir; do
   (cd "$subdir" || exit; whatever; ...)
 done

Forcing a SubShell here causes the `cd` to occur only in the subshell; for the next iteration of the loop, we're back to our normal location, regardless of whether the `cd` succeeded or failed. We don't have to change back manually, and we aren't stuck in a neverending string of `... && ...` logic preventing the use of other conditionals. The subshell version is simpler and cleaner (albeit a tiny bit slower).

Another approach is to `cd` unconditionally to where we're supposed to be, at the start of each loop iteration:

.

 here=$PWD
 find ... -type d -print0 | while IFS= read -r -d '' subdir; do
    cd "$here" || continue
    cd "$subdir" || continue
    whatever
    ...
 done

At least this way, we can `continue` to the next loop iteration and don't have to string an indefinite series of `&&` together to ensure that we reach the `cd` at the end of the loop body.

<<Anchor(pf20)>>

[ bar

"$foo" ] == The `==` operator is not valid for the POSIX `[` command. Use `=` or the [[BashFAQ/031|keyword instead.

.

 [ bar = "$foo" ] && echo yes
 [[ bar == $foo ]] && echo yes

In Bash, `[ "$x" == y ]` is accepted as an extension, which often leads Bash programmers to think it's the correct syntax. It's not; it's a Bashism. If you're going to use Bashisms, you might as well just use `[[` instead.

<<Anchor(pf21)>>

for i in {1..10}; do ./something &; done

You cannot put a `;` immediately after an `&`. Just remove the extraneous `;` entirely.

.

 for i in {1..10}; do ./something & done

Or:

.

 for i in {1..10}; do
   ./something &
 done

`&` already functions as a command terminator, just like `;` does. And you cannot mix the two.

In general, a `;` can be replaced by a newline, but not all newlines can be replaced by `;`.

<<Anchor(pf22)>>

cmd1 && cmd2 || cmd3

Some people try to use `&&` and `||` as a shortcut syntax for `if ... then ... else ... fi`, perhaps because they think they are being clever. For instance,

.

 # WRONG!
 [[ -s $errorlog ]] && echo "Uh oh, there were some errors." || echo "Successful."

However, this construct is not completely equivalent to `if ... fi` in the general case. The command that comes after the `&&` also generates an exit status, and if that exit status isn't "true" (0), then the command that comes after the `||` will also be invoked. For example:

.

 i=0
 true && ((i++)) || ((i--))  # WRONG!
 echo "$i"                   # Prints 0

What happened here? It looks like `i` should be 1, but it ends up 0. Why? Because both the `i++` and the `i--` were executed. The `((i++))` command has an exit status, and that exit status is derived from a C-like evaluation of the expression inside the parentheses. That expression's value happens to be 0 (the initial value of `i`), and in C, an expression with an integer value of 0 is considered false. So `((i++))` (when `i` is 0) has an exit status of 1 (false), and therefore the `((i--))` command is executed as well.

Another clever person thinks that we can fix it by using the pre-increment operator, since the exit status from `++i` (with `i` initially 0) is true:

.

 i=0
 true && (( ++i )) || (( --i ))  # STILL WRONG!
 echo "$i"                       # Prints 1 by dumb luck

But that's missing the point of the example. It just happens to work by coincidence, and you cannot rely on `x && y || z` if `y` has any chance of failure! (This example still fails if we initialize `i` to -1 instead of 0.)

If you need safety, or if you simply aren't sure how this works, or if anything in the preceding paragraphs wasn't completely clear, please just use the simple `if ... fi` syntax in your programs.

.

 i=0
 if true; then
   ((i++))
 else
   ((i--))
 fi
 echo "$i"    # Prints 1

This section also applies to Bourne shell, here is the code that illustrates it:

.

 # WRONG!
 true && { echo true; false; } || { echo false; true; }

Output is two lines "true" and "false", instead the single line "true".

<<Anchor(pf23)>>

echo "Hello World!"

The problem here is that, in an interactive Bash shell (in versions prior to 4.3), you'll see an error like:

.

 bash: !": event not found

This is because, in the default settings for an interactive shell, Bash performs csh-style history expansion using the exclamation point. This is not a problem in shell scripts; only in interactive shells.

Unfortunately, the obvious attempt to "fix" this won't work:

.

 $ echo "hi\!"
 hi\!

The easiest solution is unsetting the `histexpand` option: this can be done with `set +H` or `set +o histexpand`

. Question: Why is playing with `histexpand` more apropriate than single quotes?
. I personally ran into this issue when I was manipulating song files, using commands like

mp3info -t "Don't Let It Show" ...
mp3info -t "Ah! Leah!" ...

Using single quotes is extremely inconvenient because of all the songs with apostrophes in their titles. Using double quotes ran into the history expansion issue. (And imagine a file that has both in its name. The quoting would be atrocious.) Since I never actuallyusehistory expansion, my personal preference was to turn it off in `~/.bashrc`. -- GreyCat

These solutions will work:

.

 echo 'Hello World!'

.

 set +H
 echo "Hello World!"

.

 histchars=

Many people simply choose to put `set +H` or `set +o histexpand` in their `~/.bashrc` to deactivate history expansion permanently. This is a personal preference, though, and you should choose whatever works best for you.

Another solution is:

.

 exmark='!'
 echo "Hello, world$exmark"

In Bash 4.3 and newer, a double quote following `!` no longer triggers history expansion, but history expansion is still performed within double quotes, so while `echo "Hello World!"` is OK, these will still be a problem:

.

 echo "Hello, World!(and the rest of the Universe)"
 echo "foo!'bar'"

<<Anchor(pf24)>>

for arg in $*

Bash (like all Bourne shells) has a special syntax for referring to the list of positional parameters one at a time, and `$*` isn't it. Neither is `$@`. Both of those expand to the list of words in your script's parameters, not to each parameter as a separate word.

The correct syntax is:

.

 for arg in "$@"

 # Or simply:
 for arg

Since looping over the positional parameters is such a common thing to do in scripts, `for arg` defaults to `for arg in "$@"`. The double-quoted `"$@"` is special magic that causes each parameter to be used as a single word (or a single loop iteration). It's what you should be using at least 99% of the time.

Here's an example:

.

 # Incorrect version
 for x in $*; do
   echo "parameter: '$x'"
 done

 $ ./myscript 'arg 1' arg2 arg3
 parameter: 'arg'
 parameter: '1'
 parameter: 'arg2'
 parameter: 'arg3'

It should have been written:

.

 # Correct version
 for x in "$@"; do
   echo "parameter: '$x'"
 done
# or better:
 for x do
   echo "parameter: '$x'"
 done

 $ ./myscript 'arg 1' arg2 arg3
 parameter: 'arg 1'
 parameter: 'arg2'
 parameter: 'arg3'

<<Anchor(pf25)>>

function foo()

This works in some shells, but not in others. You should never combine the keyword `function` with the parentheses `()` when defining a function.

Bash (at least some versions) will allow you to mix the two. Most of the shells won't accept that (zsh 4.x and perhaps above will - for example). Some shells will accept `function foo`, but for maximum portability, you should always use:

.

 foo() {
  ...
 }

<<Anchor(pf26)>>

echo "~"

Tilde expansion only applies when '~' is unquoted. In this example echo writes '~' to stdout, rather than the path of the user's home directory.

Quoting path parameters that are expressed relative to a user's home directory should be done using $HOME rather than '~'. For instance consider the situation where $HOME is "/home/my photos".

.

 "~/dir with spaces" # expands to "~/dir with spaces"
 ~"/dir with spaces" # expands to "~/dir with spaces"
 ~/"dir with spaces" # expands to "/home/my photos/dir with spaces"
 "$HOME/dir with spaces" # expands to "/home/my photos/dir with spaces"

<<Anchor(pf27)>>

== local varname=$(command) == When declaring a local variable in a function, the `local` acts as a command in its own right. This can sometimes interact oddly with the rest of the line -- for example, if you wanted to capture the exit status (`$?`) of the CommandSubstitution, you can't do it. `local`'s exit status masks it.

It's best to use separate commands for this:

.

 local varname
 varname=$(command)
 rc=$?

This is also true of `export`, which will similarly mask the exit status.

The next pitfall describes another issue with this syntax:

<<Anchor(pf28)>>

== export foo=~/bar == Tilde expansion (with or without a username) is only guaranteed to occur when the tilde appears at the beginning of a word, either by itself or followed by a slash. It is also guaranteed to occur when the tilde appears immediately after the `=` in an assignment.

However, the `export` and `local` commands do not constitute an assignment. So, in some shells (like Bash), `export foo=~/bar` will undergo tilde expansion; in others (like dash), it will not.

.

 foo=~/bar; export foo    # Right!
 export foo="$HOME/bar"   # Right!

<<Anchor(pf29)>>

sed 's/$foo/good bye/'

In single quotes, bash parameter expansions like `$foo` do not get expanded. That is the purpose of single quotes, to protect characters like `$` from the shell.

Change the quotes to double quotes:

.

 foo="hello"; sed "s/$foo/good bye/"

But keep in mind, if you use double quotes you might need to use more escapes. See the Quotes page.

<<Anchor(pf30)>>

tr [A-Z] [a-z]

There are (at least) three things wrong here. The first problem is that `[A-Z]` and `[a-z]` are seen as globs by the shell. If you don't have any single-lettered filenames in your current directory, it'll seem like the command is correct; but if you do, things will go wrong. Probably at 0300 hours on a weekend.

The second problem is that this is not really the correct notation for `tr`. What this actually does is translate '[' into '['; anything in the range A-Z into a-z; and ']' into ']'. So you don't even need those brackets, and the first problem goes away.

The third problem is that depending on the locale, A-Z or a-z may not give you the 26 ASCII characters you were expecting. In fact, in some locales z is in the middle of the alphabet! The solution to this depends on what you want to happen:

.

 # Use this if you want to change the case of the 26 latin letters
 LC_COLLATE=C tr A-Z a-z

 # Use this if you want the case conversion to depend upon the locale, which might be more like what a user is expecting
 tr '[:upper:]' '[:lower:]'

The quotes are required on the second command, to avoid globbing.

<<Anchor(pf31)>>

ps ax | grep gedit

The fundamental problem here is that the name of a running process is inherently unreliable. There could be more than one legitimate gedit process. There could be something else disguising itself as gedit (changing the reported name of an executed command is trivial). For real answers to this, see ProcessManagement.

The following is the quick and dirty stuff.

Searching for the PID of (for example) gedit, many people start with

$ ps ax | grep gedit
10530 ?        S      6:23 gedit
32118 pts/0    R+     0:00 grep gedit

which, depending on a RaceCondition, often yields grep itself as a result. To filter grep out:

ps ax | grep -v grep | grep gedit   # will work, but ugly

An alternative to this is to use:

ps ax | grep '[g]edit'              # quote to avoid shell GLOB

This will ignore the grep itself in the process table as that is [g]edit and grep is looking for gedit once evaluated.

On GNU/Linux, the parameter -C can be used instead to filter by commandname:

$ ps -C gedit
  PID TTY          TIME CMD
10530 ?        00:06:23 gedit

But why bother when you could just use pgrep instead?

$ pgrep gedit
10530

Now in a second step the PID is often extracted by awk or cut:

$ ps -C gedit | awk '{print $1}' | tail -n1

but even that can be handled by some of the trillions of parameters for ps:

$ ps -C gedit -opid=
10530

If you're stuck in 1992 and aren't using pgrep, you could use the ancient, obsolete, deprecated pidof (GNU/Linux only) instead:

$ pidof gedit
10530

and if you need the PID to kill the process, pkill might be interesting for you. Note however that, for example, `pgrep/pkill ssh` would also find processes named sshd, and you wouldn't want to kill those.

Unfortunately some programs aren't started with their name, for example firefox is often started as firefox-bin, which you would need to find out with - well - ps ax | grep firefox. :) Or, you can stick with pgrep by adding some parameters:

$ pgrep -fl firefox
3128 /usr/lib/firefox/firefox
7120 /usr/lib/firefox/plugin-container /usr/lib/flashplugin-installer/libflashplayer.so -greomni /usr/lib/firefox/omni.ja 3128 true plugin

Please read ProcessManagement. Seriously.

<<Anchor(pf32)>>

printf "$foo"

This isn't wrong because of quotes, but because of a format string exploit. If `$foo` is not strictly under your control, then any `\` or `%` characters in the variable may cause undesired behavior.

Always supply your own format string:

printf %s "$foo"
printf '%s\n' "$foo"

<<Anchor(pf33)>>

for i in {1..$n}

The BashParser performs BraceExpansion before any other expansions or substitutions. So the brace expansion code sees the literal `$n`, which is not numeric, and therefore it doesn't expand the curly braces into a list of numbers. This makes it nearly impossible to use brace expansion to create lists whose size is only known at run-time.

Do this instead:

for ((i=1; i<=n; i++)); do
...
done

In the case of simple iteration over integers, an arithmetic `for` loop should almost always be preferred over brace expansion to begin with, because brace expansion pre-expands every argument which can be slower and unnecessarily consumes memory.

<<Anchor(pf34)>>

if $foo = $bar (depending on intent)

When the right-hand side of an `=` operator inside [[BashFAQ/031| is not quoted, bash does pattern matching against it, instead of treating it as a string. So, in the code above, if `bar` contains `*`, the result will always be true. If you want to check for equality of strings, the right-hand side should be quoted:

if [[ $foo = "$bar" ]]

If you want to do pattern matching, it might be wise to choose variable names that indicate the right-hand side contains a pattern. Or use comments.

It's also worth pointing out that if you quote the right-hand side of `=~` it also forces a simple string comparison, rather than a regular expression matching. This leads us to:

<<Anchor(pf35)>>

if $foo =~ 'some RE'

The quotes around the right-hand side of the `=~` operator cause it to become a string, rather than a RegularExpression. If you want to use a long or complicated regular expression and avoid lots of backslash escaping, put it in a variable:

re='some RE'
if [[ $foo =~ $re ]]

This also works around the difference in how `=~` works across different versions of bash. Using a variable avoids some nasty and subtle problems.

The same problem occurs with pattern matching inside ``: <pre> [[ $foo = "*.glob" # Wrong! *.glob is treated as a literal string. $foo = *.glob # Correct. *.glob is treated as a glob-style pattern.

<<Anchor(pf36)>>

[ -n $foo ] or [ -z $foo ]

When using the `[` command, you must quote each substitution that you give it. Otherwise, `$foo` could expand to 0 words, or 42 words, or any number of words that isn't 1, which breaks the syntax.

[ -n "$foo" ]
[ -z "$foo" ]
[ -n "$(some command with a "$file" in it)" ]

# [[ doesn't perform word-splitting or glob expansion, so you could also use:
[[ -n $foo ]]
[[ -z $foo ]]

<<Anchor(pf37)>>

-e "$broken_symlink" returns 1 even though $broken_symlink exists

Test follows symlinks, therefore if a symlink is broken, i.e. it points to a file that doesn't exists or is in a directory you don't have access to, test -e returns 1 for it even though it exists.

In order to work around it (and prepare against it) you should use:

# bash/ksh/zsh
[[ -e "$broken_symlink" || -L "$broken_symlink" ]]

# POSIX sh+test
[ -e "$broken_symlink" ] || [ -L "$broken_symlink" ]

<<Anchor(pf38)>>

ed file <<<"g/d\{0,3\}/s//e/g" fails

The problem caused because ed doesn't accept 0 for \{0,3\}.

You can check that the following do work:

ed file <<<"g/d\{1,3\}/s//e/g"

Note that this happens even though POSIX states that BRE (which is the Regular Expression flavor used by ed) should accept 0 as the minimum number of occurrences (see section 5).

<<Anchor(pf39)>>

expr sub-string fails for "match"

This works reasonably well

most of the time

word=abcde
expr "$word" : ".\(.*\)"
bcde

But WILL fail for the word "match"

word=match
expr "$word" : ".\(.*\)"

The problem is "match" is a keyword. Solution (GNU only) is prefix with a '+'

word=match
expr + "$word" : ".\(.*\)"
atch

Or, y'know, stop using `expr`. You can do everything `expr` does by using Parameter Expansion. What's that thing up there trying to do? Remove the first letter of a word? That can be done in POSIX shells using PE or Substring Expansion:

$ word=match
$ echo "${word#?}"    # PE
atch
$ echo "${word:1}"    # SE
atch

Seriously, there's no excuse for using `expr` unless you're on Solaris with its non-POSIX-conforming `/bin/sh`. It's an external process, so it's much slower than in-process string manipulation. And since nobody uses it, nobody understands what it's doing, so your code is obfuscated and hard to maintain.

<<Anchor(pf40)>>

On UTF-8 and Byte-Order Marks (BOM)

In general: Unix UTF-8 text does not use BOM. The encoding of plain text is determined by the locale or by mime types or other metadata. While the presence of a BOM would not normally damage a UTF-8 document meant only for reading by humans, it is problematic (often syntactically illegal) in any text file meant to be interpreted by automated processes such as scripts, source code, configuration files, and so on. Files starting with BOM should be considered equally foreign as those with MS-DOS linebreaks.

In shell scripting: 'Where UTF-8 is used transparently in 8-bit environments, the use of a BOM will interfere with any protocol or file format that expects specific ASCII characters at the beginning, such as the use of "#!" of at the beginning of Unix shell scripts.' http://unicode.org/faq/utf_bom.html#bom5

<<Anchor(pf41)>>

== content=$(<file) == There isn't anything wrong with this expression, but you should be aware that command substitutions (all forms:

`...`

, `$(...)`, `$(<file)`,

`<file`

, and `${ ...; }` (ksh)) remove any trailing newlines. This is often inconsequential or even desirable, but if you must preserve the literal output including any possible trailing newlines, it gets tricky because you have no way of knowing whether the output had them or how many. One ugly but usable workaround is to add a postfix inside the command substitution and remove it on the outside:

absolute_dir_path_x=$(readlink -fn -- "$dir_path"; printf x)
absolute_dir_path=${absolute_dir_path_x%x}

A less portable but arguably prettier solution is to use `read` with an empty delimiter.

# Ksh (or bash 4.2+ with lastpipe enabled)
readlink -fn -- "$dir_path" | IFS= read -rd '' absolute_dir_path

The downside to this method is that the `read` will always return false unless the command outputs a NUL byte causing only part of the stream to be read. The only way to get the exit status of the command is through `PIPESTATUS`. You could also intentionally output a NUL byte to force `read` to return true, and use `pipefail`.

set -o pipefail
{ readlink -fn -- "$dir_path"; printf '\0x'; } | IFS= read -rd '' absolute_dir_path

This is somewhat of a portability mess, as Bash supports both `pipefail` and `PIPESTATUS`, ksh93 supports `pipefail` only, and only recent versions of mksh support `pipefail`, while earlier versions supported `PIPESTATUS` only. Additionally, a bleeding-edge ksh93 version is required in order for `read` to stop at the NUL byte.

<<Anchor(pf42)>>

for file in ./* ; do if $file != .

One way to prevent programs from interpreting filenames passed to them as options is to use pathnames (see pitfall #3 above). For files under the current directory, names may be prefixed with a relative pathname `./`.

In the case of a pattern like `*.*` however, problems can arise because it matches a string of the form `./filename`. In a simple case, you can just use the glob directly to generate the desired matches. If however a separate pattern-matching step is required (e.g. the results have been preprocessed and stored in an array, and need to be filtered), it could be solved by taking the prefix into account in the pattern: `$file != ./*.*`, or by stripping the pattern from the match.

# Bash
shopt -s nullglob
for path in ./*; do
    [[ ${path##*/} != *.* ]] && rm "$path"
done

# Or even better
for file in *; do
    [[ $file != *.* ]] && rm "./$file"
done

# Or better still
for file in *.*; do
    rm "./$file"
done

Another possibility is to signal the end of options with a `--` argument. (Again, covered in #pf3).

shopt -s nullglob
for file in *; do
    [[ $file != *.* ]] && rm -- "$file"
done

<<Anchor(pf43)>>

somecmd 2>&1 >>logfile

This is by far the most common mistake involving redirections, typically performed by someone wanting to direct both stdout and stderr to a file or pipe will try this and not understand why stderr is still showing up on their terminal. If you're perplexed by this, you probably don't understand how redirections or possibly file descriptors work to begin with. Redirections are evaluated left-to-right before the command is executed. This semantically incorrect code essentially means: "first redirect standard error to where standard out is currently pointing (the tty), then redirect standard out to logfile". This is backwards. Standard error is already going to the tty. Use the following instead:

somecmd >>logfile 2>&1

See a more in-depth explanation, Copy descriptor explained, and BashGuide - redirection.

<<Anchor(pf44)>>

cmd; (( ! $? )) || die

`$?` is only required if you need to retrieve the exact status of the previous command. If you only need to test for success or failure (any non-zero status), just test the command directly. e.g.:

if cmd; then
    ...
fi

Checking an exit status against a list of alternatives might follow a pattern like this:

cmd
status=$?
case $status in
    0)
        echo success >&2
        ;;
    1)
        echo 'Must supply a parameter, exiting.' >&2
        exit 1
        ;;
    *)
        echo "Unknown error $status, exiting." >&2
        exit "$status"
esac

<<Anchor(pf45)>>

== y=$(( array[$x] )) == Due to the POSIX wording of arithmetic expansion (which calls for expansion of command substitutions after parameter expansion), expansion of an array subscript inside an arithmetic expansion can lead to code injection exploits.

Yeah, that's a lot of big, confusing words. Here's how it breaks:

$ x='$(date >&2)'        # redirection is just so we can see everything happen
$ y=$((array[$x]))       # array doesn't even have to exist
Mon Jun  2 10:49:08 EDT 2014

Quoting `"$x"` won't help, either:

$ y=$((array["$x"]))
Mon Jun  2 10:51:03 EDT 2014

The two tricks that do work are:

# 1. Escape the $x so it isn't expanded prematurely.
$ y=$((array[\$x]))

# 2. Use the full ${array[$x]} syntax.
$ y=$((${array[$x]}))

<<Anchor(pf46)>>

read num; echo $((num+1))

Always validate your input (see BashFAQ/054) before using num in an arithmetic context as it allows code injection.

$ echo 'a[$(echo injection >&2)]' | bash -c 'read num; echo $((num+1))'
injection
1

<<Anchor(pf47)>>

== IFS=, read -ra fields <<< "$csv_line" == Unbelievable as it may seem, POSIX requires the treatment of IFS as a field terminator, rather than a field separator. What this means in our example is that if there's an empty field at the end of the input line, it will be discarded:

$ IFS=, read -ra fields <<< "a,b,"
$ declare -p fields
declare -a fields='([0]="a" [1]="b")'

Where did the empty field go? It was eaten for historical reasons ("because it's always been that way"). This behavior is not unique to bash; all conformant shells do it. A non-empty field is properly scanned:

$ IFS=, read -ra fields <<< "a,b,c"
$ declare -p fields
declare -a fields='([0]="a" [1]="b" [2]="c")'

So, how do we work around this nonsense? As it turns out, appending an IFS character to the end of the input string will force the scanning to work. If there was a trailing empty field, the extra IFS character "terminates" it so that it gets scanned. If there was a trailing non-empty field, the IFS character creates a new, empty field that gets dropped.

$ input="a,b,"
$ IFS=, read -ra fields <<< "$input,"
$ declare -p fields
declare -a fields='([0]="a" [1]="b" [2]="")'

<<Anchor(pf48)>>

== export CDPATH=.:~/myProject == Do not export CDPATH.

Setting CDPATH in .bashrc is not an issue, but exporting it will cause any bash or sh script you run, that happen to use `cd`, to potentially change behaviour.

There are two problems. A script that does the following:

cd some/dir || exit
cmd to be run in some/dir

may change directory to `~/myProject/some/dir` instead of `./some/dir`, depending on what directories exist at the time. So the `cd` may succeed and take the script to the wrong directory, with potentially harmful effects of the following commands which now run in a different directory than intended.

The second problem is when `cd` is run in a context where the output is captured:

output=$(cd some/dir && some command)

As a side-effect when CDPATH is set, `cd` will output something like `/home/user/some/dir` to stdout to indicate that it found a directory through CDPATH, which in turn will end up in the output variable along with the intended output of `some command`.

A script can make itself immune to a CDPATH inherited from the environment by always prepending `./` to relative paths, or run `unset CDPATH` at the start of the script, but don't assume every scripter has taken this pitfall into account, so don't export CDPATH.

<<Anchor(pf49)>>

== OIFS="$IFS"; ...; IFS="$OIFS" == Directly assigning a variable's value to a temporary variable isn't alone enough to restore its state. The assignment will always result in a set but empty temporary variable even if the initial variable was unset. This is a particular problem for IFS because an empty IFS has a completely different meaning from an unset IFS, and setting IFS to a temporary value for a command or two is a common requirement.

An easy workaround is to designate a prefix to distinguish set from unset vars, then strip it when finished.

# oIFS is unset or null implies IFS is null.

typeset oIFS=${IFS+_${IFS}}
IFS=/; echo "${array[*]}"
${oIFS:+'false'} unset -v IFS || IFS=${oIFS#_}

A local variable is usually preferable when possible.

f() {
  local IFS
  IFS=/; echo "${array[*]}"
}
f

Subshells are another possibility.

( IFS=/; echo "${array[*]}" )

<<Anchor(pf50)>>

== hosts=( $(aws ...) ) == It is not safe to populate an array with a raw `$(...)` CommandSubstitution. The output of the command undergoes word splitting (on all whitespace, even ones that are inside quotes) and then globbing. If there's a word like `*` or `eh?` or `[abc]` in the result, it will be expanded based on filenames in the current working directory.

To select a replacement, you need to know whether the command writes its output on a single line, or multiple lines. If it's a single line:

read -ra hosts < <(aws ...)

If it's multiple lines (and you're targeting bash 4.0 or later):

readarray -t hosts < <(aws ...)

If it's multiple lines (and you want compatibility with bash 3.x, or want your command's exit status to be reflected in success or failure of the `read` operation without depending on behavior only available in bash 4.4 and newer):

IFS=$'\n' read -r -d '' -a hosts < <(aws ... && printf '\0')

This will prevent globbing. It still won't help you if you needed to avoid splitting on quoted whitespace, but unfortunately nothing bash can do handles that case. For generalized CSV (comma-separated value) file handling, you really need to switch to a language that has a dedicated CSV input library.

<<Anchor(pf51)>>

Non-atomic writes with xargs -P

GNU `xargs` supports running multiple jobs in parallel. `-P n` where n is the number of jobs to run in parallel.

seq 100 | xargs -n1 -P10 echo "$a" | grep 5
seq 100 | xargs -n1 -P10 echo "$a" > myoutput.txt

This will work fine for many situations but has a deceptive flaw: If `$a` contains more than ~1000 characters, the `echo` may not be atomic (it may be split into multiple `write()` calls), and there is a risk that two lines will be mixed.

$ perl -e 'print "a"x2000, "\n"' > foo
$ strace -e write bash -c 'read -r foo < foo; echo "$foo"' >/dev/null
write(1, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 1008) = 1008
write(1, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 993) = 993
+++ exited with 0 +++

Obviously the same issue arises if there are multiple calls to `echo` or `printf`:

slowprint() {
  printf 'Start-%s ' "$1"
  sleep "$1"
  printf '%s-End\n' "$1"
}
export -f slowprint
seq 10 | xargs -n1 -I {} -P4 bash -c "slowprint {}"
# Compare to no parallelization
seq 10 | xargs -n1 -I {} bash -c "slowprint {}"
# Be sure to see the warnings in the next Pitfall!

Outputs from the parallel jobs are mixed together, because each job consists of two (or more) separate `write()` calls.

If you need the outputs unmixed, it is therefore recommended to use a tool that guarantees output will be serialized (such as GNU Parallel).

For further details see a demonstration of the mixing problem.

<<Anchor(pf52)>>

find . -exec sh -c 'echo {}' \;

This command contains a CodeInjection vulnerability. The filename that is found by `find` is injected into a shell command and parsed by `sh`. If the filename contains shell metacharacters like `;` or `$( ... )` then the filename may be executed as code by `sh'.

The "slowprint" example in the previous Pitfall would have been a CodeInjection bug if the input weren't guaranteed to be integers.

To be more precise, POSIX find does not specify whether an argument which contains more than just `{}` is expanded. GNU `find` allows this CodeInjection to occur. Other implementations choose a safer path:

# uname -a
HP-UX imadev B.10.20 A 9000/785 2008897791 two-user license
# find /dev/null -exec sh -c 'echo {}' \;
{}

The correct approach is to separate the filename argument from the script argument:

find . -exec sh -c 'echo "$1"' x {} \;

<<Anchor(pf53)>>

sudo mycmd > /myfile

Redirection is done before the command is executed. Usually that doesn't matter, but with `sudo` we have a command being executed as a different user than the redirection.

If the redirection must be executed with `sudo`-granted privileges, then you need a wrapper:

sudo sh -c 'mycmd > /myfile'

<<Anchor(pf54)>>

sudo ls /foo/*

This is very similar to the previous pitfall. Globbing is also done before the command is executed. If the directory isn't readable by your normal user privileges, then you may need the globbing to be done in a shell that has the `sudo`-granted privileges:

sudo sh -c 'ls /foo/*'

<<Anchor(pf55)>>

myprogram 2>&-

Do not close stdin, stdout or stderr as a "shorthand" for redirecting to `/dev/null`. Write it out correctly.

myprogram 2>/dev/null

Why? Consider what happens when your program tries to write an error message to stderr. If stderr has been redirected to `/dev/null`, the write succeeds, and your program is free to carry on, secure in the knowledge that it has diligently reported the error condition.

But if stderr has been closed, then the write will fail. At that point, your program may do something unpredictable. It may carry on and ignore the failure, or it may immediately exit, considering the execution environment so broken that it cannot safely continue. Or whatever else the programmer decided the program should do when its world has become a dystopian hell.

All programs are assured that stdin, stdout and stderr will exist and will be readable/writable in an appropriate and reasonable manner. By closing one of them, you have violated your promise to this program. This is not acceptable.

Of course, an even better solution would be to actually log the errors somewhere, so you can go back and read them and figure out what's wrong.

<<Anchor(pf56)>>

Using xargs without -0

`xargs` splits on whitespace. This is unfortunate because whitespace is allowed in filenames and commonly used by GUI users. `xargs` also treats `'` and `"` specially, which can also lead to problems:

touch Dad\'s\ \"famous\"\ 1\'\ pizza.txt
touch Dad\'s\ 12\"\ records.txt
touch 2\"x1\'\ wood.txt
touch 2\"x4\"\ wood.txt

Here `xargs` warns:

# Do not do this
$ find . -type f | xargs wc
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option

Here `xargs` does not warn at all:

# Do not do this
echo * | xargs wc
find *famous* -type f | xargs wc
find *4* -type f | xargs wc

Instead use `xargs -0`:

# Do this instead
printf '%s\0' * | xargs -0 wc
find . -type f -name '*famous*' -print0 | xargs -0 wc
find . -type f -name '*4*' -exec wc {} +

If using `-0` is not simple, an alternative is to use GNU Parallel, which splits on \n. And while \n is also allowed in filenames they never occur unless your users are malicious. In any case: If you use `xargs` without `-0` put a comment in your code explaining why that is safe in your particular situation.

CategoryShell CategoryBashguide

Linux algemeen & CLI

File permissions

One of the important things you can do to help secure your MediaWiki install, is ensure that the user you are running php as (often www-data if using debian) and the user you are running mysql as, does not have write access to any web accessible directory with php enabled.

On unix-like systems, you can do this by ensuring that the mediawiki directory/files are owned by someone other than your web server user (www-data) or mysql server user. Depending on how you installed MediaWiki this may already be the case, but if not can be accomplished by doing chown -R /path/to/MediaWiki/ where username is a user other than the webserver or mysql user (commonly you would use your own username provided mysql</> and php are not running as your username).

After doing that step, you may however need to change the owner of the image directory back to the php user, as uploaded files need to go there, so MediaWiki needs to be able to write there (e.g. chown -R www-data /path/to/MediaWiki/images).
Next you run chmod -R go-w /path/to/MediaWiki to remove write access from all other users besides the file owners.
After doing that step you may need to re-enable write access to the images directory.

Directories that MediaWiki needs write access to (such as $wgCacheDirectory if that feature is enabled) should be located outside of the web root. The exception being the images directory, which must be in the web root.
However, it is important to disable php in the images directory. The details on how to do this varies with webserver, but on apache it can sometimes be accomplished by using php_flag engine off in a .htaccess</> file. If you do accomplish this via a config file in the images directory itself, you should ensure the config file is not writable by the webserver. See the section below on upload security for more details.

Your LocalSettings.php file must be readable by the php user, however it should not be world readable, to prevent other processes from discovering your database password and other sensitive information. Like all MediaWiki files, the php user should not be able to write to LocalSettings.php.

permissions

New install of MediaWiki 1.28 on Ubuntu 16.04. Permissions for /var/www/html set to 755, owned by me user:www-data.

Did a Download from Git install, before and after doing composer install, the file and directory permissions are readable and writable but not executable for group. Should I set them to 755 manually?

In your MediaWiki directory, the following SSH commands should work:

find . -type f -exec chmod 644 {} \;
find . -type d -exec chmod 755 {} \;

securing

The following picks up on a fresh working LAMP installation under Red Hat 7 or clone (CentOS 7, Scientific Linux 7, Orcale 7, etc). Set Selinux to permissive for the installation.

setenforce 0

First get the Mediawiki version you want from https://releases.wikimedia.org/mediawiki/ , at time of writing latest is https://releases.wikimedia.org/mediawiki/1.27/mediawiki-1.27.0.tar.gz and unpack it in /var/www/html/w.

Navigate to https://www.example.com/w and follow on-screen instructions to generate content used for LocalSettings.php. Create LocalSettings.php with

vi /var/www/html/w/LocalSettings.php

and paste content into file (i -> enter insert mode, CTRL+SHIFT+v to paste content, ESC -> to exit insert mode, ZZ (twice letter Z) to save and exit vi). Now secure LocalSettings.php with

chown root:apache /var/www/html/w/LocalSettings.php
chmod 640 /var/www/html/w/LocalSettings.php

Delete mw-config if it exists, since it is only used for first time setup of mediawiki.

rm -rf /var/www/html/w/mw-config

Enable use of .htaccess files by creating custom configuration file for Apache httpd.

   cat >> /etc/httpd/conf.d/custom.conf << EOF
   <Directory "/var/www/html/w">
    AllowOverride All

   EOF

Now one should customize LocalSettings.php to one's taste. Here an overview of variables that can be customized: https://www.mediawiki.org/wiki/Manual:Configuration_settings

Since we want to access our Mediawiki installation under https://www.example.com/wiki we need to set $wgArticlePath in LocalSettings.php. Just add the following line a the bottom of LocalSettings.php

$wgArticlePath = "/wiki/$1";

and update /etc/httpd/conf.d/ssl.conf by adding one line.

<VirtualHost _default_:443>
Alias /wiki /var/www/html/w/index.php # <-- only add this line

selinux

Now finish securing the Mediawiki installation. This *may* be needed for Selinux, e.g. database on different server, etc.

setsebool -P httpd_can_network_connect 1
setsebool -P httpd_can_network_connect_db 1

This *is* needed for Selinux to run Mediawiki

setsebool -P httpd_builtin_scripting 1
setsebool -P httpd_execmem 1

Set userrights and special Selinux rights, so Apache httpd has read access, but other users beside root don't.

   chown -R root:apache /var/www/html/
   find /var/www/html/w -type d -exec chmod 750 {} \;
   find /var/www/html/w -type f -exec chmod 640 {} \;

Mediawiki writes to images and cache, so they need special write premissions.

   chown -R apache:apache /var/www/html/w/images
   chown -R apache:apache /var/www/html/w/cache
   semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/w/cache(/.*)?"
   semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/w/images(/.*)?"
   restorecon -R /var/www/html/w

Found Selinux complaining about hugetlbfs, so add an exception.

   cd /var/log/audit
   grep hugetlbfs audit.log | audit2allow -M hugetlbfs
   semodule -i hugetlbfs.pp

Now restart Apache httpd and set Selinux back to enforcing.

setenforce 1
systemctl restart httpd

Understandably this covers only the basics and Mediawiki offers thousands of ways to customize it further to one's taste and security needs.

Don't forget to make regular backups.

Further suggestions can be found here https://www.pozzo-balbi.com/help/Mediawiki .

Linux algemeen & CLI

Disable cpu core for light work

# echo 0 | sudo tee /sys/bus/cpu/devices/cpu$COREID/online # Disable a CPU core to save battery and lower thermals. Works greats if your laptop is noisy and you're only doing light work

Linux algemeen & CLI

Rescan SCSI bus

# echo "0 0 0" > /sys/class/scsi_host/host5/scan

5 is changeable; it is the connection

Linux algemeen & CLI

How to send email from command line in fixed font

To send email that is in fixed font you can use:

 # (echo "Subject: bla"; echo "MIME-Version: 1.0"; echo "Content-Type: text/html"; echo "Content-Disposition: inline"; echo '<html><body><pre>The code element<SLASHpre></body></html>') |  sendmail joop@hotmail.com

Mind the /pre here.

Linux algemeen & CLI

Rsync root volume

# rsync -aAXv --exclude 'data' --exclude 'mnt' --exclude 'proc' --exclude 'sys' --exclude 'dev' --exclude 'run' /* /mnt/mmcblk/3/

Linux algemeen & CLI

Edit PDF metadata

View PDF metadata:

# exiftool file.pdf

View specific metadata:

# exiftool -Creator file.pdf
# find . -type f -iname '*.pdf' -exec exiftool -Creator {} \;

Set metadata:

# exiftool -Creator='barbapappa' file.pdf
# find . -type f -iname '*.pdf' -exec exiftool -Creator='barbapappa' {} \;

to preserve date/time:

-P

to delete backup file that is created:

-delete_original[!]

# find . -type f -iname '*.pdf' -exec exiftool -P -overwrite_original -Producer='www.bladiebla.nl' -Creator='barbapappa' {} \;
# find . -type f -iname '*.pdf' -exec exiftool -P -delete_original! {} \;

Edit in one metadatafile

Make a parameter file

-Title=Mijn Titel
-Author=Mijn Naam
-Creator=Microsoft Word
-Producer=Adobe PDF Library 15.0
-Subject=Korte samenvatting
-Keywords=tag1, tag2, tag3
-CreateDate=2025:08:14 08:00:00
-ModifyDate=2025:08:14 08:00:00

and apply with:

# exiftool -@ meta.args -overwrite_original pdfbestand.pdf

workflow

#!/bin/bash
set -e

echo  alle odg naar pdf zetten
libreoffice --headless --convert-to pdf *.odg

echo alle pdf files in 1 groot document
pdfunite {1..22}.pdf samengevoegd.pdf

echo pdftk om fouten weg te krijgen
pdftk samengevoegd.pdf output rc1.pdf

echo de metadata erin
exiftool -@ metadata.csv rc1.pdf

echo haal oude metadata weg en houd alleen de laatste huidige
qpdf --linearize rc1.pdf rc2.pdf

echo
echo De file rc.2 is de output
echo

Linux algemeen & CLI

Rsync root volume to new harddisk

# rsync -aAXvx --progress --exclude='.snapshots/' / [mountpoint of new harddisk]

Linux algemeen & CLI

Exiftool

To fix the date time on the pixel 6a photo and video files, use for photos:

# exiftool '-FileName<DateTimeOriginal' -d '%Y%m%d_%H%M%S%%-c.%%e' *.jpg

and for video files: (NOT ENTIRELY CORRECT NEEDS UPDATING!!)

# exiftool '-FileName<FileModifyDate' -d '%Y%m%d_%H%M%S%%-c.%%e' *.mp4

Linux algemeen & CLI

Lsmod

lsmod
Module                  Size  Used by
fuse                  106496  1
tcp_diag               16384  0
inet_diag              20480  1 tcp_diag
xt_CHECKSUM            16384  1
iptable_mangle         16384  1
ipt_MASQUERADE         16384  3
nf_nat_masquerade_ipv4    16384  1 ipt_MASQUERADE
iptable_nat            16384  1
nf_nat_ipv4            16384  1 iptable_nat
nf_nat                 24576  2 nf_nat_ipv4,nf_nat_masquerade_ipv4
tun                    32768  1
br_netfilter           24576  0
bridge                139264  1 br_netfilter
stp                    16384  1 bridge
llc                    16384  2 stp,bridge
ebtable_filter         16384  0
ebtables               36864  1 ebtable_filter
nf_log_ipv6            16384  20
xt_comment             16384  26
nf_log_ipv4            16384  20
nf_log_common          16384  2 nf_log_ipv4,nf_log_ipv6
xt_LOG                 16384  40
xt_limit               16384  40
af_packet              45056  0
iscsi_ibft             16384  0
iscsi_boot_sysfs       20480  1 iscsi_ibft
ip6t_REJECT            16384  3
nf_reject_ipv6         16384  1 ip6t_REJECT
nf_conntrack_ipv6      20480  9
nf_defrag_ipv6         36864  1 nf_conntrack_ipv6
ipt_REJECT             16384  5
nf_reject_ipv4         16384  1 ipt_REJECT
xt_pkttype             16384  7
xt_tcpudp              16384  84
iptable_filter         16384  1
ip6table_mangle        16384  0
nf_conntrack_netbios_ns    16384  0
nf_conntrack_broadcast    16384  1 nf_conntrack_netbios_ns
nf_conntrack_ipv4      16384  12
nf_defrag_ipv4         16384  1 nf_conntrack_ipv4
ip_tables              24576  3 iptable_filter,iptable_mangle,iptable_nat
xt_conntrack           16384  20
nf_conntrack          118784  8 nf_conntrack_netbios_ns,nf_nat,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_broadcast,nf_conntrack_ipv4,nf_conntrack_ipv6
ip6table_filter        16384  1
ip6_tables             28672  2 ip6table_filter,ip6table_mangle
x_tables               40960  17 xt_pkttype,ip6table_filter,ip6table_mangle,xt_comment,xt_CHECKSUM,ip_tables,xt_tcpudp,ipt_MASQUERADE,xt_limit,xt_conntrack,xt_LOG,iptable_filter,ebtables,ipt_REJECT,iptable_mangle,ip6_tables,ip6t_REJECT
msr                    16384  0
nct6775                61440  0
hwmon_vid              16384  1 nct6775
joydev                 20480  0
hid_logitech_hidpp     20480  0
xfs                  1085440  1
kvm_amd                86016  0
kvm                   610304  1 kvm_amd
raid456               147456  1
irqbypass              16384  1 kvm
async_raid6_recov      20480  1 raid456
async_memcpy           16384  2 raid456,async_raid6_recov
libcrc32c              16384  2 xfs,raid456
crct10dif_pclmul       16384  0
async_pq               16384  2 raid456,async_raid6_recov
async_xor              16384  3 async_pq,raid456,async_raid6_recov
crc32_pclmul           16384  0
xor                    20480  1 async_xor
crc32c_intel           24576  1
async_tx               16384  5 async_pq,raid456,async_xor,async_memcpy,async_raid6_recov
hid_logitech_dj        20480  0
ghash_clmulni_intel    16384  0
raid6_pq              118784  3 async_pq,raid456,async_raid6_recov
md_mod                155648  2 raid456
drbg                   28672  1
ansi_cprng             16384  0
usbhid                 53248  0
snd_usb_audio         188416  0
snd_usbmidi_lib        36864  1 snd_usb_audio
snd_hwdep              16384  1 snd_usb_audio
snd_rawmidi            36864  1 snd_usbmidi_lib
snd_seq_device         16384  1 snd_rawmidi
snd_pcm               135168  1 snd_usb_audio
aesni_intel           167936  0
aes_x86_64             20480  1 aesni_intel
lrw                    16384  1 aesni_intel
gf128mul               16384  1 lrw
glue_helper            16384  1 aesni_intel
ablk_helper            16384  1 aesni_intel
cryptd                 20480  3 ghash_clmulni_intel,aesni_intel,ablk_helper
snd_timer              36864  1 snd_pcm
snd                    90112  7 snd_usb_audio,snd_hwdep,snd_timer,snd_pcm,snd_rawmidi,snd_usbmidi_lib,snd_seq_device
'''r8169                  90112  0'''
soundcore              16384  1 snd
'''mii                    16384  1 r8169'''
i2c_piix4              24576  0
k10temp                16384  0
pcspkr                 16384  0
shpchp                 36864  0
acpi_cpufreq           20480  0
fjes                   32768  0
video                  45056  0
processor              49152  5 acpi_cpufreq
button                 16384  0
nfsd                  348160  9
auth_rpcgss            65536  1 nfsd
nfs_acl                16384  1 nfsd
lockd                 102400  1 nfsd
grace                  16384  2 nfsd,lockd
sunrpc                364544  13 nfsd,auth_rpcgss,lockd,nfs_acl
ext4                  651264  3
crc16                  16384  1 ext4
jbd2                  118784  1 ext4
mbcache                16384  4 ext4
sr_mod                 24576  0
cdrom                  61440  1 sr_mod
sd_mod                 57344  11
amdkfd                143360  1
amd_iommu_v2           20480  1 amdkfd
ohci_pci               16384  0
'''r8168                 540672  0'''
radeon               1597440  1
i2c_algo_bit           16384  1 radeon
drm_kms_helper        167936  1 radeon
syscopyarea            16384  1 drm_kms_helper
ahci                   40960  6
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
fb_sys_fops            16384  1 drm_kms_helper
libahci                36864  1 ahci
ttm                   110592  1 radeon
ohci_hcd               53248  1 ohci_pci
pata_atiixp            16384  2
ehci_pci               16384  0
xhci_pci               16384  0
ehci_hcd               81920  1 ehci_pci
xhci_hcd              192512  1 xhci_pci
usbcore               274432  9 snd_usb_audio,ohci_hcd,ohci_pci,snd_usbmidi_lib,ehci_hcd,ehci_pci,usbhid,xhci_hcd,xhci_pci
drm                   397312  4 ttm,drm_kms_helper,radeon
usb_common             16384  1 usbcore
ata_generic            16384  0
pata_amd               20480  0
libata                274432  5 ahci,libahci,pata_amd,ata_generic,pata_atiixp
sg                     49152  0
dm_multipath           32768  0
dm_mod                131072  1 dm_multipath
scsi_dh_rdac           20480  0
scsi_dh_emc            16384  0
scsi_dh_alua           20480  0
scsi_mod              253952  8 sg,scsi_dh_alua,scsi_dh_rdac,dm_multipath,scsi_dh_emc,libata,sd_mod,sr_mod
autofs4                45056  2
</nowiki>

Linux algemeen & CLI

Lspci

lspci
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Root Complex
00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Richland [Radeon HD 8570D]
00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Root Port
00:10.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB XHCI Controller (rev 09)
00:10.1 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB XHCI Controller (rev 09)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller (rev 11)
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller (rev 11)
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller (rev 11)
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller (rev 11)
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 16)
00:14.1 IDE interface: Advanced Micro Devices, Inc. [AMD] FCH IDE Controller
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 11)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD] FCH PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller (rev 11)
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 5
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 11)

Linux algemeen & CLI

Linux commandline tips 2

Command
•	apropos whatis
•	man -t man \| ps2pdf - > man.pdf
	which command
	time command
•	time cat
•	nice info
•	renice 19 -p $$
dir navigation
•	cd -
•	cd
	(cd dir && command)
•	pushd .
file searching
•	alias l='ls -l --color=auto'
•	ls -lrt
•	ls /usr/bin \| pr -T9 -W$COLUMNS
	find -name '*.[ch]' \| xargs grep -E 'expr'
	find -type f -print0 \| xargs -r0 grep -F 'example'
	find -maxdepth 1 -type f \| xargs grep -F 'example'
	find -maxdepth 1 -type d \| while read dir; do echo $dir; echo cmd2; done
•	find -type f ! -perm -444
•	find -type d ! -perm -111
•	locate -r 'file[^/]*\.txt'
•	look reference
•	grep --color reference /usr/share/dict/words
archives and compression
	gpg -c file
	gpg file.gpg
	tar -c dir/ \| bzip2 > dir.tar.bz2
	bzip2 -dc dir.tar.bz2 \| tar -x
	tar -c dir/ \| gzip \| gpg -c \| ssh user@remote 'dd of=dir.tar.gz.gpg'
	find dir/ -name '*.txt' \| tar -c --files-from=- \| bzip2 > dir_txt.tar.bz2
	find dir/ -name '*.txt' \| xargs cp -a --target-directory=dir_txt/ --parents
	( tar -c /dir/to/copy ) \| ( cd /where/to/ && tar -x -p )
	( cd /dir/to/copy && tar -c . ) \| ( cd /where/to/ && tar -x -p )
	( tar -c /dir/to/copy ) \| ssh -C user@remote 'cd /where/to/ && tar -x -p'
	dd bs=1M if=/dev/sda \| gzip \| ssh user@remote 'dd of=sda.gz'
rsync (Network efficient file copier: Use the --dry-run option for testing)
	rsync -P rsync://rsync.server.com/path/to/file file
	rsync --bwlimit=1000 fromfile tofile
	rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html'
	rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/
ssh (Secure SHell)
	ssh $USER@$HOST command
•	ssh -f -Y $USER@$HOSTNAME xeyes
	scp -p -r $USER@$HOST: file dir/
	ssh -g -L 8080:localhost:80 root@$HOST
	ssh -R 1434:imap:143 root@$HOST
wget (multi purpose download tool)
•	(cd dir/ && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html)
	wget -c http://www.example.com/large.file
	wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/
	wget ftp://remote/file[1-9].iso/
•	wget -q -O- http://www.pixelbeat.org/timeline.html \| grep 'a href' \| head
	echo 'wget url' \| at 01:00
	wget --limit-rate=20k url
	wget -nv --spider --force-html -i bookmarks.html
	wget --mirror http://www.example.com/
networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete)
	ethtool eth0
	ethtool --change eth0 autoneg off speed 100 duplex full
	iwconfig eth1
	iwconfig eth1 rate 1Mb/s fixed
•	iwlist scan
•	ip link show
	ip link set dev eth0 name wan
	ip link set dev eth0 up
•	ip addr show
	ip addr add 1.2.3.4/24 brd + dev eth0
•	ip route show
	ip route add default via 1.2.3.254
•	tc qdisc add dev lo root handle 1:0 netem delay 20msec
•	tc qdisc del dev lo root
•	host pixelbeat.org
•	hostname -i
•	whois pixelbeat.org
•	netstat -tupl
•	netstat -tup
windows networking (Note samba is the package that provides all this windows specific networking support)
•	smbtree
	nmblookup -A 1.2.3.4
	smbclient -L windows_box
	mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/share
	echo 'message' \| smbclient -M windows_box
text manipulation (Note sed uses stdin and stdout. Newer versions support inplace editing with the -i option)
	sed 's/string1/string2/g'
	sed 's/.*1/\12/g'
	sed '/ #/d; /^ $/d'
	sed ':a; /\\$/N; s/\\\n//; ta'
	sed 's/[ \t]*$//'
	sed 's/$[`"$\]$/\\\1/g'
•	seq 10 \| sed "s/^/ /; s/ *.{7, }/\1/"
	sed -n '1000{p;q}'
	sed -n '10,20p;20q'
	sed -n 's/.<title>$.$<\/title>.*/\1/ip;T;q'
	sed -i 42d ~/.ssh/known_hosts
	sort -t. -k1,1n -k2,2n -k3,3n -k4,4n
•	echo 'Test' \| tr '[:lower:]' '[:upper:]'
•	tr -dc '[:print:]' < /dev/urandom
•	history \| wc -l
set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file)
	sort file1 file2 \| uniq
	sort file1 file2 \| uniq -d
	sort file1 file1 file2 \| uniq -u
	sort file1 file2 \| uniq -u
	join -t'\0' -a1 -a2 file1 file2
	join -t'\0' file1 file2
	join -t'\0' -v2 file1 file2
	join -t'\0' -v1 -v2 file1 file2
math
•	echo '(1 + sqrt(5))/2' \| bc -l
•	echo 'pad=20; min=64; (10010^6)/((pad+min)8)' \| bc
•	echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' \| python
•	echo 'pad=20; plot [64:1518] (100106)/((pad+x)8)' \| gnuplot -persist
•	echo 'obase=16; ibase=10; 64206' \| bc
•	echo $((0x2dec))
•	units -t '100m/9.58s' 'miles/hour'
•	units -t '500GB' 'GiB'
•	units -t '1 googol'
•	seq 100 \| (tr '\n' +; echo 0) \| bc
calendar
•	cal -3
•	cal 9 1752
•	date -d fri
•	[ $(date -d "tomorrow" +%d) = "01" ]
•	date --date='25 Dec' +%A
•	date --date='@2147483647'
•	TZ=':America/Los_Angeles' date
	echo "mail -s 'get the train' P@draigBrady.com < /dev/null" \| at 17:45
•	echo "DISPLAY=$DISPLAY xmessage cooker" \| at "NOW + 30 minutes"
locales
•	printf "%'d\n" 1234
•	BLOCK_SIZE=\'1 ls -l
•	echo "I live in `locale territory`"
•	LANG=en_IE.utf8 locale int_prefix
•	locale \| cut -d= -f1 \| xargs locale -kc \| less
recode (Obsoletes iconv, dos2unix, unix2dos)
•	recode -l \| less
	recode windows-1252.. file_to_change.txt
	recode utf-8/CRLF.. file_to_change.txt
	recode iso-8859-15..utf8 file_to_change.txt
	recode ../b64 < file.txt > file.b64
	recode /qp.. < file.txt > file.qp
	recode ..HTML < file.txt > file.html
•	recode -lf windows-1252 \| grep euro
•	echo -n 0x80 \| recode latin-9/x1..dump
•	echo -n 0x20AC \| recode ucs-2/x2..latin-9/x
•	echo -n 0x20AC \| recode ucs-2/x2..utf-8/x
CDs
	gzip < /dev/cdrom > cdrom.iso.gz
	mkisofs -V LABEL -r dir \| gzip > cdrom.iso.gz
	mount -o loop cdrom.iso /mnt/dir
	cdrecord -v dev=/dev/cdrom blank=fast
	gzip -dc cdrom.iso.gz \| cdrecord -v dev=/dev/cdrom -
	cdparanoia -B
	cdrecord -v dev=/dev/cdrom -audio *.wav
	oggenc --tracknum='track' track.cdda.wav -o 'track.ogg'
disk space (See also FSlint)
•	ls -lSr
•	du -s * \| sort -k1,1rn \| head
•	df -h
•	df -i
•	fdisk -l
•	rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' \| sort -k1,1n
•	dpkg-query -W -f='${Installed-Size;10}\t${Package}\n' \| sort -k1,1n
•	dd bs=1 seek=2TB if=/dev/null of=ext3.test
•	> file
monitoring/debugging
•	tail -f /var/log/messages
•	strace -c ls >/dev/null
•	strace -f -e open ls >/dev/null
•	ltrace -f -e getenv ls >/dev/null
•	lsof -p $$
•	lsof ~
•	tcpdump not port 22
•	ps -e -o pid,args --forest
•	ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu \| sed '/^ 0.0 /d'
•	ps -e -orss=,args= \| sort -b -k1,1n \| pr -TW$COLUMNS
•	ps -C firefox-bin -L -o pid,tid,pcpu,state
•	ps -p 1,2
•	last reboot
•	free -m
•	watch -n.1 'cat /proc/interrupts'
system information (see also sysinfo) ('#' means root access is required)
•	uname -a
•	head -n1 /etc/issue
•	cat /proc/partitions
•	grep MemTotal /proc/meminfo
•	grep "model name" /proc/cpuinfo
•	lspci -tv
•	lsusb -tv
•	mount \| column -t
•	grep -F capacity: /proc/acpi/battery/BAT0/info
#	dmidecode -q \| less
#	smartctl -A /dev/sda \| grep Power_On_Hours
#	hdparm -i /dev/sda
#	hdparm -tT /dev/sda
#	badblocks -s /dev/sda
interactive (see also linux keyboard shortcuts)
•	readline
•	screen
•	mc
•	gnuplot
•	links
•	xdg-open .
miscellaneous
•	alias hd='od -Ax -tx1z -v'
•	alias realpath='readlink -f'
•	set \| grep $USER
	touch -c -t 0304050607 file
•	python -m SimpleHTTPServer

google_ad_section_start(weight=ignore)

Linux algemeen & CLI

Linux commandline tips 3

Download a file and uncompress it while it downloads

$wget http://URL/FILE.tar.gz -O - | tar xfz -

If you play loto, try this command to generate the 6 numbers :

$echo $(shuf -n 6 -i 1-49 | sort -n)

To get the CPU temperature continuously on the desktop

$while :; do acpi -t | osd_cat -p bottom ; sleep 1; done &

Search for large files and show size and location.

$find . -size +100000k -exec du -h {} \;

Get a random Command

$ls /usr/bin | shuf -n 1

This is useful if you want to explore various random commands.

More commands :

Bash logger :Log everything from a bash script to a file

$script /tmp/log.txt

Using NMAP to check if a port is open or close

$nmap -oG - -T4 -p22 -v 192.168.0.254 | grep ssh

Get the IP of the host your coming from when logged in remotely

$echo ${SSH_CLIENT%% *}

Multimedia

Convert .flv to other movie format

How convert flv2mpg or flv to other formats:

http://www.mydigitallife.info/2006/12/14/free-online-flv-converters/ http://www.mplayerhq.hu/design7/dload.html ,example:

mencoder input.flv -ofps 15 -vf scale=300:-2 -oac lavc -ovc lavc -lavcopts vcodec=msmpeg4v2:acodec=mp3:abitrate=64 -o output.avi

Multimedia

Wine

Installing Newsleecher in Wine

As for installing your licence, I recommend exporting the registry key from your Windows install, edit the first line in it where it says 'Windows Registry Editor Version 5.00' and change it to 'REGEDIT4'.

Then open a terminal and type 'wine regedit'. In the registry editor, choose file, import and point to the exported registry file. You can also manually enter every key in the registry, copying the data from Windows to Linux.

The key is located in HKLM\Software\05970870
Search for your Licence ID. Found in Help > About. Be sure to untick 'match whole string' when searching the registry. It's 'swXXXXXXXX'.
Not all licenses start with sw. Some start with DI or NL.

Multimedia

How to make a .iso file from your CD or DVD

To make an ISO from your CD/DVD, place the media in your drive but do not mount it.

If it automounts, unmount it.

for DVD:

dd if=/dev/dvd of=dvd.iso

for CD:

dd if=/dev/cdrom of=cd.iso

SATA device

dd if=/dev/sr0 of=file.iso

SCSI device:

dd if=/dev/scd0 of=cd.iso

Multimedia

Convert an AVCHD / MTS file to MP4 using ffmpeg

Convert an .MTS file to a .MP4 file

We want to convert it to .MP4, for instance to show it on a mobile device Android, or to play it om xbmc, or to import into kdenlive. ...

For a whole directory, type:

IFS=$(echo -en "\n\b"); for i in *.MTS; do ffmpeg -i "$i" -vcodec mpeg4 -b:v 15M -acodec libmp3lame -b:a 192k "$i.mp4"; done

or offtopic:

find . -print0 | while read -d $'\0' file
do
  echo -v "$file"
done

Now, if you want to resize it for some reason (show on mobile phone):

#ffmpeg -i 00031.MTS -s 480x320 -b:v 4000k 00031.MP4

-i = input file

-s = size <=== PUT THE SIZE PARAMETER JUST AFTER THE INPUTFILENAME, OTHERWISE YOU GET AN ERROR

-b:v = videobitrate, 8000k gives a slightly better image, but doubles the filesize -b:a = audiobitrate

Using these parameters you shrink a 25MB movie to a 6 MB movie.

Convert an .DV file to a .MP4 file

ffmpeg -i tape.dv -vcodec mpeg4 -b:v 15M -acodec libmp3lame -ab 192k -threads 2 -ilme tape.mp4

-ilme = Force interlacing support in encoder (MPEG-2 and MPEG-4 only). Use this option if your input file is interlaced and you want to keep the interlaced format for minimum losses. The alternative is to deinterlace the input stream with -deinterlace, but deinterlacing introduces losses.

Convert an .MTS file to a .DV file for video editing

We want to convert it to .DV, because then our video editing program (kdenlive) will process the clips much much faster!!!

#ffmpeg -i 00005.MTS -f avi -b:v 16000k -ab 192k 00005.DV

-i = input file

-f = format, appeared neccessary

-b:v = videobitrate, this value of 16000k is exactly the same as the source file, because we don't want any loss!!

-b:a = audio bitrate, this value of 192k is exactly the same as the source file, because we don't want any loss!!

ALTERNATIVE

I also found this commandline, worked very well:

ffmpeg-kino -threads 2 -i 00120.MTS -s 720x576 -r pal -aspect 16:9 -ac 2 -ar 48000 -pix_fmt yuv420p -y 00120.MTS.dv

Another commandline:

ffmpeg -y -i 00141.MTS -vcodec mpeg4 -b:v 8000000 -ab 128000 -s 1280x720 new2.mp4
ffmpeg -y -i 00101.MTS -vcodec mpeg4 -b:v 14M -acodec ac3 -ab 192k new.mp4 <-- deze gebruikt om .MTS in kdenlive te importeren

dus:

for i in *; do ffmpeg -y -i "$i" -vcodec mpeg4 -b:v 14M -acodec ac3 -ab 192k "$i".mp4; done

Convert to WebM (VP8)

A valid WebM file can only contain VP8 video and Vorbis audio in a .webm container.

ffmpeg -i bla.mp4 -vcodec libvpx -b:v 15M -acodec libvorbis -b:a 192k out.webm

For web page (reduced size):

# ffmpeg -i 44.mp4 -s 568x320 -vcodec libvpx -b:v 300k -acodec libvorbis -b:a 64k 44.webm

Convert an .MTS file to Flash (FLV)

ffmpeg -i bla.mts -s 640x360 -ar 22050 -b:v 1M blaat.flv

of pas de bitrate aan voor hogere kwaliteit:

ffmpeg -i bla.mts -s 640x360 -ar 22050 -b:v 3M blaat.flv

Convert an old analogue letterbox recording to SD digital format

The input file was

Input #0, mpeg, from '1106_20091226203000.mpg':                                                                                                                                           
  Duration: 04:24:55.20, start: 0.276144, bitrate: 6934 kb/s
    Stream #0.0[0x1e0]: Video: mpeg2video (Main), yuv420p, 720x576 [PAR 64:45 DAR 16:9], 8000 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16, 384 kb/s

and the picture has big black borders on the top and bottom and looks squashed. It was supposed to be stretched to fill up the screen (Our old analogue TV did that automatically).

This must be done in 2 steps: first the cropping, then rescaling back to 16:9

The first command is

ffmpeg -i input.mpg -vf crop=720:430 -vcodec mpeg2video -b:v 8M -acodec mp2 -b:a 384k 25ycrop.mpg

Note that we try to keep the audio/video exactly the same as the sourcefile. The commandline options -sameq and -vcodec copy don't seem to work here.

Next we must scale the video back to full screen, again the same settings for vcodec and acodec:

ffmpeg -i 25ycrop.mpg -s 720x576 -aspect 16:9 -vcodec mpeg2video -b:v 8M -acodec mp2 -b:a 384k bla.mpg

Other examples

 # ffmpeg -i 00039.MTS -f avi -vcodec mpeg4 -b:v 16000 bla.wmv
 # ffmpeg -i 00039.MTS -f avi -vcodec msmpeg4v2 -b:v 16000 bla.wmv
 # ffmpeg -y -i VIDEO0005.3gp  -vcodec wmv2 -f avi  -ar 44100 kerst2009.avi

 # ffmpeg -formats            -> lists formats
 # ffmpeg -f avi              -> force avi format
 # ffmpeg -codecs             -> lists codecs
 # ffmpeg -vcodec mpeg4       -> use codec
 # ffmpeg -acodec libmp3lame  -> use mp3 audio codec
 # ffmpeg -i <inputfile>      -> show info about movie file
 # ffmpeg -y                  -> overwrite outputfiles
 # ffmpeg -vcodec copy -ss 00:01:00 -t 00:03:00 -i infile.mpg outfile.mpg -> this cut the file from 1 minute to 4 minutes (t=duration)
or
 # ffmpeg -i inputfilename.mpg -sameq -ss 00:01:00 -t 00:03:00 outfile.mpg
The above will extract 3 minutes of mpg file from the 1 minute mark using same quality as the source file. This is useful for Variable Bit Rate (VBR) encoded files.
 # ffmpeg -target dvd|vcd|svcd|dv|dv50  -> all options are set automagically, but you can overrule as long as they don't conflict w/ the standard
          -r  --> set framerate, default = 25. also 'pal' is accepted
          -threads --> set thread count
 # ffmpeg -i 00090.MTS -f avi -b 6000k -ab 192k -vcodec mpeg2video 00090.mpg
    convert MTS to XBOX readable format

example command to preserve EXIF data and timestamps

$ for i in *.MOV ; do ffmpeg -i "$i" -preset slow -crf 27 "${i//.MOV}.mp4" ; exiftool -tagsFromFile "$i" "${i//.MOV}.mp4" ; touch -r "$i" "${i//.MOV}.mp4" ; done

!!I should have used ${i%.MOV}.mp4 to match only the suffix pattern, not anywhere in the variable string.

example of Intel Skylake 520 vaapi encoding

# ffmpeg -threads 4 -i dcc-basf.mp4 -vaapi_device /dev/dri/renderD128 -vcodec h264_vaapi -vf format='nv12|vaapi,hwupload' output2.mp4

Burning subtitles from embedded subtitles stream into the video image

# ffmpeg -i s01e01.mkv -c:v libx264 -c:a copy -vf "subtitles=s01e01.mkv:stream_index=1" output.mkv

where

stream_index is the # of the subtitle stream

This gives encoding fps of around 30.

ultrafast encoding:

# ffmpeg -i s01e01.mkv -c:v libx264 -preset ultrafast -c:a copy -vf "subtitles=s01e01.mkv:stream_index=1" output.mkv

This gives encoding fps of around 140.

Note:

The "ultrafast" preset will likely result in noticeable quality degradation compared to slower presets. If higher quality is important, consider using faster presets like "superfast" or "veryfast" which offer a better balance between speed and quality.

For even faster encoding, you can explore hardware acceleration options like NVENC (for NVIDIA GPUs) or QuickSync (for Intel GPUs) if your system supports them.

Intel hardware encoding:

# ffmpeg -i s01e01.mkv -c:v h264_qsv -c:a copy -vf "subtitles=s01e01.mkv:stream_index=1" output2.mkv

This gives encoding fps of around 180

Another option: -preset veryfast

Sharpening

Using the -vf option.

# Strong luma sharpen effect parameters
unsharp=7:7:2.5

PvdM: dit betekent [unsharp @ 0x64f7c0] effect:sharpen type:luma msize_x:7 msize_y:7 amount:2.50

# Strong blur of both luma and chroma parameters
unsharp=7:7:-2:7:7:-2

# Use the default values with ffmpeg
./ffmpeg -i in.avi -vf "unsharp" out.mp4

PvdM: dit betekent [unsharp @ 0x64f7a0] effect:sharpen type:luma msize_x:5 msize_y:5 amount:1.00

Example:

ffmpeg -i part1.dv -vf "unsharp" -target dvd test2.mpg

PvdM: getest en werkt goed:

#ffmpeg -i out.dv -vf unsharp=9:9:1.5:9:9:1.5 out2.dv

Image Stabilizing

$ ffmpeg -i $invoer -vf vidstabdetect -f null -
$ ffmpeg -i $invoer -vf vidstabtransform=crop=black:smoothing=20,unsharp=5:5:0.8:3:3:0.4 $invoer.stabilized.mp4

This is a 2 trap proccess

Timestamps

To set the timestamp of the file to the video's internal creation date/time, you can use:

exiftool '-CreateDate>FileModifyDate' FILE

To force a date/time when converting:

ffmpeg -i 00041.MTS -vcodec mpeg4 -acodec copy -timestamp 2000012312:21:34 bla.mp4

Split video files

ffmpeg -i input.mpg -ss 00:00:10 -t 00:00:30 out1.mpg

-ss is the start point in hh:mm:ss from the beginning of your video file

-t is the length of time in hh:mm:ss of your new segment.

-to is the end time hh:mm:ss of the new segment.

So, in the above example, you're starting 10 seconds in from the beginning of the original file and ending 30 seconds later.

If you want to create multiple parts in one pass then the following should work:

ffmpeg -i input.mpg -ss 00:00:10 -t 00:00:30 out1.mpg -ss 00:00:35 -t 00:00:30 out2.mpg

In this example, the first segment is the same as the first example, but you're also creating a second file starting at 35 seconds in and being 30 seconds long.

Remember to use the correct encoding:

ffmpeg -i input.mpg -ss 00:00:10 -vcodec copy -acodec copy output.mpg

Cut an avi file from second 1 to 12.9.

mencoder -ss 00:01 -endpos 00:12.900 -ovc copy -oac copy -o out.avi in.avi

Merge video files

To merge 2 or more files, use the 'cat' command and pipe it through ffmpeg, like this:

cat 1.mpg 2.mpg | ffmpeg (-f mpeg) -i – -vcodec copy -acodec copy outfile.mpg

Unfortunately, this does not work. Again, mencoder to the rescue:

mencoder -oac copy -ovc copy -o output.mp4 1.mp4 2.mp4

Flip video files

hflip and vflip

ffmpeg -i input.mp4 -vf hflip,vflip,format=yuv420p -codec:v libx264 -preset medium -crf 23 -codec:a copy output.mkv

The above did not work, but this did:

ffmpeg -i VID_20130915_110755.mp4 -vf transpose=2,transpose=2 output.mp4

rotate video files 90 degrees counterclockwise

ffmpeg -i 20160416_152451.mp4 -vf transpose=2 -c:a copy output.mp4

create timelapse movie

To create a timelapse movie from images in a directory, use:

mencoder "mf://*.jpg" -mf fps=5 -o test.avi -ovc lavc -lavcopts vcodec=msmpeg4v2:vbitrate=1000

I know, this is mencoder, but it works.

FFmpeg uses the following format:

ffmpeg -f image2 -r 10 -b:v 1M -i %03d.jpg test2.mp4

but this doesn't work. Also it requires the imagefilenames to be of name 001.jpg - 999.jpg, which is not always the case.

Rename files sequentially

cnt=1;for i in `ls *.jpg`; do mv ${i} ${cnt}.jpg;cnt=$((cnt+1)); done

To list available formats (supported pixel formats, video formats, and frame sizes) for a particular input device:

$ ffmpeg -f v4l2 -list_formats all -i /dev/video0

Alternatively you could use

v4l2-ctl --list-formats-ext

to list available formats.

To take a picture with a webcam

ffmpeg -f video4linux2 -i /dev/video0 -vframes 1 test.jpeg

To record video with a webcam

ffmpeg -f video4linux2 -s 640x480 -r 20 -i /dev/video0 -f m4v out.m4v

-r = framerate

to stream live to youtube from IPCAM

ffmpeg -rtsp_transport tcp -i "rtsp://192.168.1.34:554/iphone/11?username:password" -f lavfi -i anullsrc=r=16000:cl=mono -c:a aac -ar 44100 -pix_fmt yuv420p -preset medium -s 1280x720 -bufsize 6000k -vb 6000k -maxrate 6000k -deinterlace -vcodec libx264 -crf 18 -r 50 -f flv -force_key_frames source -x264-params keyint=20:scenecut=0 "rtmp://a.rtmp.youtube.com/live2/

record the screen, capture the desktop

Use the x11grab device:

ffmpeg -video_size 1024x768 -framerate 25 -f x11grab -i :0.0+100,200 output.mp4

This will grab the image from desktop, starting with the upper-left corner at (x=100, y=200) with the width and height of 1024x768.

If you need audio too you can use ALSA (see Capture/ALSA for more info):

ffmpeg -video_size 1024x768 -framerate 25 -f x11grab -i :0.0+100,200 -f alsa -ac 2 -i hw:0 output.mkv

Or the pulse input device:

ffmpeg -video_size 1024x768 -framerate 25 -f x11grab -i :0.0+100,200 -f pulse -ac 2 -i default output.mkv

Record fullscreen TV image (ziggo stream):

ffmpeg -video_size 1600x900 -framerate 30 -f x11grab -i :0.0 -c:v libx264 -qp 0 -preset ultrafast -f pulse -ac 2 -i default capture.mkv

deze werkt:

ffmpeg -video_size 1920x1080 -framerate 15 -f x11grab -i :0.0 -f pulse -ac 2 -i default -c:v libx264 -qp 0 -preset ultrafast capture.mkv

Merging video and audio, with audio re-encoding

See this example, taken from this blog entry but updated for newer syntax. It should be something to the effect of:

ffmpeg -i video.mp4 -i audio.wav -c:v copy -c:a aac -strict experimental output.mp4

Here, we assume that the video file does not contain any audio stream yet, and that you want to have the same output format (here, MP4) as the input format.

The above command transcodes the audio, since MP4s cannot carry PCM audio streams. You can use any other desired audio codec if you want. See the AAC Encoding Guide for more info.

If your audio or video stream is longer, you can add the -shortest option so that ffmpeg will stop encoding once one file ends.

Copying the audio without re-encoding

If your output container can handle (almost) any codec – like MKV – then you can simply copy both audio and video streams:

ffmpeg -i video.mp4 -i audio.wav -c copy output.mkv

Replacing audio stream

If your input video already contains audio, and you want to replace it, you need to tell ffmpeg which audio stream to take:

ffmpeg -i video.mp4 -i audio.wav -c:v copy -c:a aac -strict experimental -map 0:v:0 -map 1:a:0 output.mp4

The map option makes ffmpeg only use the first video stream from the first input and the first audio stream from the second input for the output file.

extract images from a video

ffmpeg -i giveafuck.mp4 image-%3d.jpeg

-r This is used to set the frame rate of video. i.e. no. of frames to be extracted into images per second. The default value is 25, using which, would have yielded a large number of images.

Create a thumbnail image every X seconds of the video

Output a single frame from the video into an image file:

ffmpeg -i input.flv -ss 00:00:14.435 -vframes 1 out.png

This example will seek to the position of 0h:0m:14sec:435msec and output one frame (-vframes 1) from that position into a PNG file.

Output one image every second, named out1.png, out2.png, out3.png, etc.:

ffmpeg -i input.flv -vf fps=1 out%d.png

Output one image every minute, named img001.jpg, img002.jpg, img003.jpg, etc. The %03d dictates that the ordinal number of each output image will be formatted using 3 digits:

ffmpeg -i myvideo.avi -vf fps=1/60 img%03d.jpg

Output one image every ten minutes:

ffmpeg -i test.flv -vf fps=1/600 thumb%04d.bmp

Merge pictures into an Animated GIF

convert -delay x -loop 0 out*gif

In the command, "-delay" is an option that controls the animation speed. This option indicates that [ticks/ticks-per-second] seconds must elapse before the display of the next frame. The "-loop 0" option indicates infinite loops of animation. If you want, you can specify "-loop N", in which case the animation will repeat itself N times.

For example, to create an animated GIF image with 20 frames-per-second and infinite loop, use the following command.

$ convert -delay 1x20 -loop 0 out*.gif animation.gif

The last (optional) step is to reduce the size of the created GIF file, by using ImageMagick's GIF optimizer.

Use the following command to reduce the GIF size.

$ convert -layers Optimize animation.gif animation_small.gif

DVD

Rip entire DVD

# cat *.VOB | ffmpeg -i - -acodec aac -aq 100 -ac 2 -vcodec libx264 -crf 24 -threads 0 /data/pil.mp4

Re-encode for DVD use

# mythreplex --demux --fix_sync -o /data/stream -v 224 -c 128 "/data/newfile2.mpg
# tcrequant /data/stream.mv2 /data/video.small.m2v 1.17052143685

Pull Chapters from VOB

You can pull chapters from VOB files using mplayer. Here's a command line to pull chapter 3 from the DVD drive and dump it to a VOB.

mplayer dvd:// -chapter 3-3 -dumpstream -dumpfile 3.vob

physical DVD

mplayer dvd://2 -chapter 3-3 -dumpstream -dumpfile ~/3.VOB

DVD .iso image

mplayer dvd://2 -dvd-device "$dvd_iso" -chapter 3-3 -dumpstream -dumpfile ~/3.VOB

Rip VOB to mpeg4

and deinterlace as well.

ffmpeg -i VTS_01_1.VOB -vcodec mpeg4 -b:v 10M -acodec copy -deinterlace lan1deinterlaced.mp4

Rip VOB to DV

The following command will rip a VOB file strait from an unencrypted DVD and convert it to a strait DV file.

ffmpeg -i /cdrom/VIDEO_TS/VTS_01_1.VOB -target dv /home/joel/Videos/game_vi.dv

Rip VOB to DVD

The following command will rip a VOB file to an MPEG2 video with AC3 audio for a DVD. It also uses the '-sameq' option which uses the same quality factor in the encoder as in the decoder, allowing almost lossless encoding.

ffmpeg -i myfile.vob -target dvd -sameq myfile.mpg

Rip VOB to VCD

The following command will rip a VOB file to a MPEG1 video with MP1 audio.

ffmpeg -i myfile.vob -target vcd myfile.mpg

Rip VOB to Flash (FLV)

The following command will rip a VOB file to 352×240 (the same size as VCD) and will save it as a flash file.

ffmpeg -i myfile.vob -s 352x240 myfile.flv

Rip VOB to FLAC audio

The following command will extract an AUDIO-DVD files to flac. List the VOB files in order:

cat VTS_01_1.VOB VTS_01_2.VOB | ffmpeg -i - -acodec flac -vn output.flac

unfortunately, there will be no CUE file.

Audio

Adjust volume of audio file

ffmpeg -i input.wav -af "volume=5dB" output.mp3

gives a 5 dB gain

combine multiple sources to one destination

# ffmpeg -f concat -i /tmp/list.txt output3.flac

where list.txt contains:

file '/tmp/file1'
file '/tmp/file2'

or try:

ffmpeg -i "concat:input1.mpg|input2.mpg|input3.mpg" -c copy output.mpg

better still:

ffmpeg -f concat -i <( for f in *.wav; do echo "file '$(pwd)/$f'"; done ) output.wav

Convert .dsf files (SACD) to standard .wav files (PCM - CD quality)

ffmpeg -i 04.dsf -acodec pcm_s16le -ar 44100 -ac 2 output.wav

This however, does not give enough audio quality.

The Golden Line:

ffmpeg -i input.dsf -acodec pcm_s24le -ar 44100 -ac 2 44100_24.wav

I have tried it with 32 bits and with 88200 and higher, could not hear it.

copy/convert from multistream sources

Given the following input file:

# fmpeg -i input.mkv

ffmpeg version ... Copyright (c) 2000-2012 the FFmpeg developers
...
Input #0, matroska,webm, from 'input.mkv':
  Duration: 01:39:44.02, start: 0.000000, bitrate: 5793 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p, 1920x800, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Stream #0:1(ger): Audio: dts (DTS), 48000 Hz, 5.1(side), s16, 1536 kb/s (default)
    Stream #0:2(eng): Audio: dts (DTS), 48000 Hz, 5.1(side), s16, 1536 kb/s
    Stream #0:3(ger): Subtitle: text (default)

If we want to extract only audio streams, from input file, then we can do it like this:

ffmpeg -i input.mkv -map 0:1 -map 0:2 -c copy output.mkv

or use the appropriate codecs

Create multichannel WAVs or FLACs

This command will 'rip' the first 5 minutes (which is the first song actually) of a quadraphonic (4.0) .WAV file that is 45 minutes long:

ffmpeg -i The\ Doobie\ Bothers-\ Toulouse\ Street.wav -acodec pcm_s16le -b:a 1400k -ss 0:0:0 -t 0:5:0 \
-ac 4 -map_channel 0.0.0 -map_channel 0.0.1 -map_channel 0.0.4 -map_channel 0.0.5 output.wav -y

The resulting file 'output.wav' will be a multichannel wav with the following channel order:

1. front left

2. front right

3. rear left

4. rear right

When loaded into audacity, and then when exported into either .WAV or .FLAC move the channel slider to 6 channels and the following channel mix must be done:

output 1 -> channel: 1

output 2 -> channel: 2

output 3 -> channel: 5

output 4 -> channel: 6

Extract channels from multitrack sources

To extract the center channel (channel 3, counting the channels starts at 0) which is usually the solo voice and/or other interesting discrete stuff:

# ffmpeg -i yes.flac -ac 1 -map_channel 0.0.2 center.flac

Channel identification

I found this:

The track order of all 5.1 surround files follows the standard 
as defined by SMPT/EBU and implemented by SourceForge in FLAC (v. 1.2.1b): 
L – R – C – Lfe – Ls - Rs flac.sourceforge.net

Also:

In accordance with ANSI/CEA-863-A¹

Zero-based order within multi-channel mp3/wav/flac datastream	Order within DTS/AAC	Channel name	Color-coding on commercial receiver and cabling
0	1	Front left	White
1	2	Front right	Red
2	0	Center	Green
3	5	Low frequency	Purple
4	3	Surround left	Blue
5	4	Surround right	Grey
6	6	Surround back left	Brown
7	7	Surround back right	Khaki

Front left	Center	Front right
Surround left		Surround right
Surround back left		Surround back right
Low frequency

Standard speaker channels

This table shows the various speaker configurations that are commonly used for end-user equipment. The order and identifiers are those specified for the channel mask in the standard uncompressed WAV file format (which contains a raw multichannel PCM stream) and are used according to the same specification for most PC connectible digital sound hardware and PC operating systems capable of handling multiple channels. While it is certainly possible to build any speaker configuration, there isn't a lot of commercially available movie or music content for alternative speaker configurations. Such cases, however, can be worked around by remixing the source content channels to the speaker channels using a matrix table specifying how much of each content channel is played through each speaker channel.

Standard speaker channels

This table shows the various speaker configurations that are commonly used for end-user equipment. The order and identifiers are those specified for the channel mask in the standard uncompressed WAV file format (which contains a raw multichannel PCM stream) and are used according to the same specification for most PC connectible digital sound hardware and PC operating systems capable of handling multiple channels.² ³ While it is certainly possible to build any speaker configuration, there isn't a lot of commercially available movie or music content for alternative speaker configurations. Such cases, however, can be worked around by remixing the source content channels to the speaker channels using a matrix table specifying how much of each content channel is played through each speaker channel.

Downmix of multichannel 5.1 audio to real quad 4 channel four speaker

A lot of surround music today is supplied in a 5.1 channel format. That means front left, front right, center, subwoofer, rear left, and rear right channel. But if you have a quadraphonic setup like me, you only have 4 speakers, which are real speakers, so no subwoofer is present nor needed. And the center speaker is also absent. So you have left front, right front, left rear and right rear.

But then you have a problem with the voices in the surround mix, which are usually mixed on the center channel. Sometimes the vocals are ONLY mixed into the center channel. Without a center channel setup, you will hear no vocals. Sometimes vocals are duplicated in the front left and right channel in the audio mix, but then the vocals will sound thin, if present at all. So what is needed is a downmix of the center channel to the two front channels. Or in other words:

quad destination:         5.1 source:
left front                <- left front + center channel
right front               <- right front + center channel
left rear                 <- left rear
right rear                <- right rear

This can be achieved by the following ffmpeg command: (where $i is the surround file)

ffmpeg -i "$i" -af "pan=quad|c0<c0+c2|c1<c1+c2|c2=c4|c3=c5" "$i.flac"

Or, for a complete directory:

IFS=$(echo -en "\n\b"); for i in *.wav; do ffmpeg -i "$i" -af "pan=quad|c0<c0+c2|c1<c1+c2|c2=c4|c3=c5" "$i.flac"; done

Compand

ffmpeg -i input.wav -filter:a "dynaudnorm" output.wav

You will most likely need to tweak the filter options for your particular use case. Use the examples given for the compand filter as a starting point:

Make music with both quiet and loud passages suitable for listening to in a noisy environment:

compand=.3|.3:1|1:-90/-60|-60/-40|-40/-30|-20/-20:6:0:-90:0.2

Another example for audio with whisper and explosion parts:

compand=0|0:1|1:-90/-900|-70/-70|-30/-9|0/-3:6:0:0:0

Here, the second example's options are, specifically:

attacks=0|0
decays=1|1
points=-90/-900|-70/-70|-30/-9|0/-3
soft-knee=6
gain=0
volume=0
delay=0

You may adjust the gain to keep the same dynamic processing but adjust it to your baseline input level.

More examples: Make music with both quiet and loud passages suitable for listening to in a noisy environment: compand=.3|.3:1|1:-90/-60|-60/-40|-40/-30|-20/-20:6:0:-90:0.2 Another example for audio with whisper and explosion parts:

compand=0|0:1|1:-90/-900|-70/-70|-30/-9|0/-3:6:0:0:0 A noise gate for when the noise is at a lower level than the signal:

compand=.1|.1:.2|.2:-900/-900|-50.1/-900|-50/-50:.01:0:-90:.1

Here is another noise gate, this time for when the noise is at a higher level than the signal (making it, in some ways, similar to squelch):

compand=.1|.1:.1|.1:-45.1/-45.1|-45/-900|0/-900:.01:45:-90:.1

2:1 compression starting at -6dB:

compand=points=-80/-80|-6/-6|0/-3.8|20/3.5

2:1 compression starting at -9dB:

compand=points=-80/-80|-9/-9|0/-5.3|20/2.9

2:1 compression starting at -12dB:

compand=points=-80/-80|-12/-12|0/-6.8|20/1.9

2:1 compression starting at -18dB:

compand=points=-80/-80|-18/-18|0/-9.8|20/0.7

3:1 compression starting at -15dB:

compand=points=-80/-80|-15/-15|0/-10.8|20/-5.2

Compressor/Gate:

compand=points=-80/-105|-62/-80|-15.4/-15.4|0/-12|20/-7.6

Expander:

compand=attacks=0:points=-80/-169|-54/-80|-49.5/-64.6|-41.1/-41.1|-25.8/-15|-10.8/-4.5|0/0|20/8.3

Hard limiter at -6dB:

compand=attacks=0:points=-80/-80|-6/-6|20/-6

Hard limiter at -12dB:

compand=attacks=0:points=-80/-80|-12/-12|20/-12

Hard noise gate at -35 dB:

compand=attacks=0:points=-80/-115|-35.1/-80|-35/-35|20/20

Soft limiter:

compand=attacks=0:points=-80/-80|-12.4/-12.4|-6/-8|0/-6.8|20/-2.8

HTC Hero

For the HTC Hero (Android) the default movie properties are:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '26112009004.mp4':
  Duration: 00:00:39.06, start: 0.000000, bitrate: 424 kb/s
    Stream #0.0(und): Video: mpeg4, yuv420p, 320x240 [PAR 1:1 DAR 4:3], 372 kb/s, 15 tbr, 30k tbn, 30k tbc
    Stream #0.1(und): Audio: aac, 48000 Hz, mono, s16, 48 kb/s
  Metadata
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp423gp4isom

HTC Desire

Screen size: 800x480 For the Desire, the default movie properties are:

Convert .MTS files for Desire:

for i in * ; do ffmpeg -i $i -s 800x480 -b 8000k /media/disk/DCIM/oostenrijk\ videos/$i.MP4; done

Samsung Galaxy S Plus

Screen size: 800x480

Encode videos for Samsung Galaxy S Plus

ffmpeg -i "27.mp4" -s 800x480 -vcodec mpeg4 -b 768k -acodec libmp3lame -ab 128k -dmix_mode loro "27.mobile.mp4"

where

512k gives rather bad image
1M gives perfect image
768k hits the sweetspot
-dmix_mode loro is used for downmixing surround sound

Codecs

most used video codecs that ffmpeg can encode

WARNING!!! DEPRECATED !!!!!!!

Codecs:                                                                                                                                         
 D..... = Decoding supported                                                                                                                    
 .E.... = Encoding supported                                                                                                                    
 ..V... = Video codec                                                                                                                           
 ..A... = Audio codec                                                                                                                           
 ..S... = Subtitle codec                                                                                                                        
 ...S.. = Supports draw_horiz_band                                                                                                              
 ....D. = Supports direct rendering method 1                                                                                                    
 .....T = Supports weird frame truncation                                                                                                       
 ------   
 dvvideo         DV (Digital Video)
 flashsv         Flash Screen Video
 flc             Flash Video (FLV) / Sorenson Spark / Sorenson H.263
 libtheora       libtheora Theora
 libx264         libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
 libxvid         libxvidcore MPEG-4 part 2
 mjpeg           MJPEG (Motion JPEG)
 mpeg1video      MPEG-1 video
 mpeg2video      MPEG-2 video
 mpeg4           MPEG-4 part 2
 msmpeg4         MPEG-4 part 2 Microsoft variant version 3                                                            
 msmpeg4v1       MPEG-4 part 2 Microsoft variant version 1                                                            
 msmpeg4v2       MPEG-4 part 2 Microsoft variant version 2
 wmv1            Windows Media Video 7
 wmv2            Windows Media Video 8

all video codecs that ffmpeg can encode

Codecs:
 D..... = Decoding supported
 .E.... = Encoding supported
 ..V... = Video codec
 ..A... = Audio codec
 ..S... = Subtitle codec
 ...I.. = Intra frame-only codec
 ....L. = Lossy compression
 .....S = Lossless compression
 -------
 D.VI.. 012v                 Uncompressed 4:2:2 10-bit
 D.V.L. 4xm                  4X Movie
 D.VI.S 8bps                 QuickTime 8BPS video
 .EVIL. a64_multi            Multicolor charset for Commodore 64 (encoders: a64multi )
 .EVIL. a64_multi5           Multicolor charset for Commodore 64, extended with 5th color (colram) (encoders: a64multi5 )
 D.V..S aasc                 Autodesk RLE
 D.VIL. aic                  Apple Intermediate Codec
 DEVIL. amv                  AMV Video
 D.V.L. anm                  Deluxe Paint Animation
 D.V.L. ansi                 ASCII/ANSI art
 DEVIL. asv1                 ASUS V1
 DEVIL. asv2                 ASUS V2
 D.VIL. aura                 Auravision AURA
 D.VIL. aura2                Auravision Aura 2
 D.V... avrn                 Avid AVI Codec
 DEVI.. avrp                 Avid 1:1 10-bit RGB Packer
 D.V.L. avs                  AVS (Audio Video Standard) video
 DEVI.. avui                 Avid Meridien Uncompressed
 DEVI.. ayuv                 Uncompressed packed MS 4:4:4:4
 D.V.L. bethsoftvid          Bethesda VID video
 D.V.L. bfi                  Brute Force & Ignorance
 D.V.L. binkvideo            Bink video
 D.VI.. bintext              Binary text
 DEVI.S bmp                  BMP (Windows and OS/2 bitmap)
 D.V..S bmv_video            Discworld II BMV video
 D.VI.S brender_pix          BRender PIX image
 D.V.L. c93                  Interplay C93
 D.V.L. cavs                 Chinese AVS (Audio Video Standard) (AVS1-P2, JiZhun profile)
 D.V.L. cdgraphics           CD Graphics video
 D.VIL. cdxl                 Commodore CDXL video
 D.V.L. cinepak              Cinepak
 DEVIL. cljr                 Cirrus Logic AccuPak
 D.VI.S cllc                 Canopus Lossless Codec
 D.V.L. cmv                  Electronic Arts CMV video (decoders: eacmv )
 D.V... cpia                 CPiA video format
 D.V..S cscd                 CamStudio (decoders: camstudio )
 D.VIL. cyuv                 Creative YUV (CYUV)
 D.V.L. dfa                  Chronomaster DFA
 DEV.LS dirac                Dirac (decoders: dirac libschroedinger ) (encoders: libschroedinger )
 DEVIL. dnxhd                VC3/DNxHD
 DEVI.S dpx                  DPX image
 D.V.L. dsicinvideo          Delphine Software International CIN video
 DEVIL. dvvideo              DV (Digital Video)
 D.V..S dxa                  Feeble Files/ScummVM DXA
 D.VI.S dxtory               Dxtory
 D.V.L. escape124            Escape 124
 D.V.L. escape130            Escape 130
 D.VILS exr                  OpenEXR image
 DEV..S ffv1                 FFmpeg video codec #1
 DEVI.S ffvhuff              Huffyuv FFmpeg variant
 DEV..S flashsv              Flash Screen Video v1
 DEV.L. flashsv2             Flash Screen Video v2
 D.V..S flic                 Autodesk Animator Flic video
 DEV.L. flv1                 FLV / Sorenson Spark / Sorenson H.263 (Flash Video) (decoders: flv ) (encoders: flv )
 D.V..S fraps                Fraps
 D.VI.S frwu                 Forward Uncompressed
 D.V.L. g2m                  Go2Meeting
 DEV..S gif                  GIF (Graphics Interchange Format)
 DEV.L. h261                 H.261
 DEV.L. h263                 H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2
 D.V.L. h263i                Intel H.263
 DEV.L. h263p                H.263+ / H.263-1998 / H.263 version 2
 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_vdpau ) (encoders: libx264 libx264rgb )
 DEVI.S huffyuv              HuffYUV
 D.V.L. idcin                id Quake II CIN video (decoders: idcinvideo )
 D.VI.. idf                  iCEDraw text
 D.V.L. iff_byterun1         IFF ByteRun1 (decoders: iff )
 D.V.L. iff_ilbm             IFF ILBM (decoders: iff )
 D.V.L. indeo2               Intel Indeo 2
 D.V.L. indeo3               Intel Indeo 3
 D.V.L. indeo4               Intel Indeo Video Interactive 4
 D.V.L. indeo5               Intel Indeo Video Interactive 5
 D.V.L. interplayvideo       Interplay MVE video
 DEVILS jpeg2000             JPEG 2000
 DEVILS jpegls               JPEG-LS
 D.VIL. jv                   Bitmap Brothers JV video
 D.V.L. kgv1                 Kega Game Video
 D.V.L. kmvc                 Karl Morton's video codec
 D.VI.S lagarith             Lagarith lossless
 .EVI.S ljpeg                Lossless JPEG
 D.VI.S loco                 LOCO
 D.V.L. mad                  Electronic Arts Madcow Video (decoders: eamad )
 D.VIL. mdec                 Sony PlayStation MDEC (Motion DECoder)
 D.V.L. mimic                Mimic
 DEVIL. mjpeg                Motion JPEG
 D.VIL. mjpegb               Apple MJPEG-B
 D.V.L. mmvideo              American Laser Games MM Video
 D.V.L. motionpixels         Motion Pixels video
 DEV.L. mpeg1video           MPEG-1 video (decoders: mpeg1video mpeg1video_vdpau )
 DEV.L. mpeg2video           MPEG-2 video (decoders: mpeg2video mpegvideo mpegvideo_vdpau )
 DEV.L. mpeg4                MPEG-4 part 2 (decoders: mpeg4 mpeg4_vdpau ) (encoders: mpeg4 libxvid )
 D.V.L. mpegvideo_xvmc       MPEG-1/2 video XvMC (X-Video Motion Compensation)
 D.V.L. msa1                 MS ATC Screen
 D.V.L. msmpeg4v1            MPEG-4 part 2 Microsoft variant version 1
 DEV.L. msmpeg4v2            MPEG-4 part 2 Microsoft variant version 2
 DEV.L. msmpeg4v3            MPEG-4 part 2 Microsoft variant version 3 (decoders: msmpeg4 ) (encoders: msmpeg4 )
 D.V..S msrle                Microsoft RLE
 D.V.L. mss1                 MS Screen 1
 D.VIL. mss2                 MS Windows Media Video V9 Screen
 DEV.L. msvideo1             Microsoft Video 1
 D.VI.S mszh                 LCL (LossLess Codec Library) MSZH
 D.V.L. mts2                 MS Expression Encoder Screen
 D.VIL. mvc1                 Silicon Graphics Motion Video Compressor 1
 D.VIL. mvc2                 Silicon Graphics Motion Video Compressor 2
 D.V.L. mxpeg                Mobotix MxPEG video
 D.V.L. nuv                  NuppelVideo/RTJPEG
 D.V.L. paf_video            Amazing Studio Packed Animation File Video
 DEVI.S pam                  PAM (Portable AnyMap) image
 DEVI.S pbm                  PBM (Portable BitMap) image
 DEVI.S pcx                  PC Paintbrush PCX image
 DEVI.S pgm                  PGM (Portable GrayMap) image
 DEVI.S pgmyuv               PGMYUV (Portable GrayMap YUV) image
 D.VIL. pictor               Pictor/PC Paint
 DEV..S png                  PNG (Portable Network Graphics) image
 DEVI.S ppm                  PPM (Portable PixelMap) image
 DEVIL. prores               Apple ProRes (iCodec Pro) (decoders: prores prores_lgpl ) (encoders: prores prores_aw prores_ks )
 D.VIL. ptx                  V.Flash PTX image
 D.VI.S qdraw                Apple QuickDraw
 D.V.L. qpeg                 Q-team QPEG
 DEV..S qtrle                QuickTime Animation (RLE) video
 DEVI.S r10k                 AJA Kona 10-bit RGB Codec
 DEVI.S r210                 Uncompressed RGB 10-bit
 DEVI.S rawvideo             raw video
 D.VIL. rl2                  RL2 video
 DEV.L. roq                  id RoQ video (decoders: roqvideo ) (encoders: roqvideo )
 D.V.L. rpza                 QuickTime video (RPZA)
 DEV.L. rv10                 RealVideo 1.0
 DEV.L. rv20                 RealVideo 2.0
 D.V.L. rv30                 RealVideo 3.0
 D.V.L. rv40                 RealVideo 4.0
 D.V.L. sanm                 LucasArts SMUSH video
 DEVI.S sgi                  SGI image
 D.VI.S sgirle               SGI RLE 8-bit
 D.V.L. smackvideo           Smacker video (decoders: smackvid )
 D.V.L. smc                  QuickTime Graphics (SMC)
 D.V... smv                  Sigmatel Motion Video (decoders: smvjpeg )
 DEV.LS snow                 Snow
 D.VIL. sp5x                 Sunplus JPEG (SP5X)
 DEVI.S sunrast              Sun Rasterfile image
 DEV.L. svq1                 Sorenson Vector Quantizer 1 / Sorenson Video 1 / SVQ1
 D.V.L. svq3                 Sorenson Vector Quantizer 3 / Sorenson Video 3 / SVQ3
 DEVI.S targa                Truevision Targa image
 D.VI.. targa_y216           Pinnacle TARGA CineWave YUV16
 D.V.L. tgq                  Electronic Arts TGQ video (decoders: eatgq )
 D.V.L. tgv                  Electronic Arts TGV video (decoders: eatgv )
 DEV.L. theora               Theora (encoders: libtheora )
 D.VIL. thp                  Nintendo Gamecube THP video
 D.V.L. tiertexseqvideo      Tiertex Limited SEQ video
 DEVI.S tiff                 TIFF image
 D.VIL. tmv                  8088flex TMV
 D.V.L. tqi                  Electronic Arts TQI video (decoders: eatqi )
 D.V.L. truemotion1          Duck TrueMotion 1.0
 D.V.L. truemotion2          Duck TrueMotion 2.0
 D.V..S tscc                 TechSmith Screen Capture Codec (decoders: camtasia )
 D.V.L. tscc2                TechSmith Screen Codec 2
 D.VIL. txd                  Renderware TXD (TeXture Dictionary) image
 D.V.L. ulti                 IBM UltiMotion (decoders: ultimotion )
 DEVI.S utvideo              Ut Video
 DEVI.S v210                 Uncompressed 4:2:2 10-bit
 D.VI.S v210x                
 DEVI.. v308                 Uncompressed packed 4:4:4
 DEVI.. v408                 Uncompressed packed QT 4:4:4:4
 DEVI.S v410                 Uncompressed 4:4:4 10-bit
 D.V.L. vb                   Beam Software VB
 D.VI.S vble                 VBLE Lossless Codec
 D.V.L. vc1                  SMPTE VC-1 (decoders: vc1 vc1_vdpau )
 D.V.L. vc1image             Windows Media Video 9 Image v2
 D.VIL. vcr1                 ATI VCR1
 D.VIL. vixl                 Miro VideoXL (decoders: xl )
 D.V.L. vmdvideo             Sierra VMD video
 D.V..S vmnc                 VMware Screen Codec / VMware Video
 D.V.L. vp3                  On2 VP3
 D.V.L. vp5                  On2 VP5
 D.V.L. vp6                  On2 VP6
 D.V.L. vp6a                 On2 VP6 (Flash version, with alpha channel)
 D.V.L. vp6f                 On2 VP6 (Flash version)
 DEV.L. vp8                  On2 VP8 (decoders: vp8 libvpx ) (encoders: libvpx )
 ..V.L. vp9                  Google VP9
 D.V.L. webp                 WebP
 DEV.L. wmv1                 Windows Media Video 7
 DEV.L. wmv2                 Windows Media Video 8
 D.V.L. wmv3                 Windows Media Video 9 (decoders: wmv3 wmv3_vdpau )
 D.V.L. wmv3image            Windows Media Video 9 Image
 D.VIL. wnv1                 Winnov WNV1
 D.V.L. ws_vqa               Westwood Studios VQA (Vector Quantized Animation) video (decoders: vqavideo )
 D.V.L. xan_wc3              Wing Commander III / Xan
 D.V.L. xan_wc4              Wing Commander IV / Xxan
 D.VI.. xbin                 eXtended BINary text
 DEVI.S xbm                  XBM (X BitMap) image
 DEVIL. xface                X-face image
 DEVI.S xwd                  XWD (X Window Dump) image
 DEVI.. y41p                 Uncompressed YUV 4:1:1 12-bit
 D.V.L. yop                  Psygnosis YOP Video
 DEVI.. yuv4                 Uncompressed packed 4:2:0
 D.V..S zerocodec            ZeroCodec Lossless Video
 DEVI.S zlib                 LCL (LossLess Codec Library) ZLIB
 DEV..S zmbv                 Zip Motion Blocks Video
 D.A.L. 8svx_exp             8SVX exponential
 D.A.L. 8svx_fib             8SVX fibonacci
 DEA.L. aac                  AAC (Advanced Audio Coding) (encoders: aac libvo_aacenc )
 D.A.L. aac_latm             AAC LATM (Advanced Audio Coding LATM syntax)
 DEA.L. ac3                  ATSC A/52A (AC-3) (encoders: ac3 ac3_fixed )
 D.A.L. adpcm_4xm            ADPCM 4X Movie
 DEA.L. adpcm_adx            SEGA CRI ADX ADPCM
 D.A.L. adpcm_afc            ADPCM Nintendo Gamecube AFC
 D.A.L. adpcm_ct             ADPCM Creative Technology
 D.A.L. adpcm_dtk            ADPCM Nintendo Gamecube DTK
 D.A.L. adpcm_ea             ADPCM Electronic Arts
 D.A.L. adpcm_ea_maxis_xa    ADPCM Electronic Arts Maxis CDROM XA
 D.A.L. adpcm_ea_r1          ADPCM Electronic Arts R1
 D.A.L. adpcm_ea_r2          ADPCM Electronic Arts R2
 D.A.L. adpcm_ea_r3          ADPCM Electronic Arts R3
 D.A.L. adpcm_ea_xas         ADPCM Electronic Arts XAS
 DEA.L. adpcm_g722           G.722 ADPCM (decoders: g722 ) (encoders: g722 )
 DEA.L. adpcm_g726           G.726 ADPCM (decoders: g726 ) (encoders: g726 )
 D.A.L. adpcm_ima_amv        ADPCM IMA AMV
 D.A.L. adpcm_ima_apc        ADPCM IMA CRYO APC
 D.A.L. adpcm_ima_dk3        ADPCM IMA Duck DK3
 D.A.L. adpcm_ima_dk4        ADPCM IMA Duck DK4
 D.A.L. adpcm_ima_ea_eacs    ADPCM IMA Electronic Arts EACS
 D.A.L. adpcm_ima_ea_sead    ADPCM IMA Electronic Arts SEAD
 D.A.L. adpcm_ima_iss        ADPCM IMA Funcom ISS
 D.A.L. adpcm_ima_oki        ADPCM IMA Dialogic OKI
 DEA.L. adpcm_ima_qt         ADPCM IMA QuickTime
 D.A.L. adpcm_ima_rad        ADPCM IMA Radical
 D.A.L. adpcm_ima_smjpeg     ADPCM IMA Loki SDL MJPEG
 DEA.L. adpcm_ima_wav        ADPCM IMA WAV
 D.A.L. adpcm_ima_ws         ADPCM IMA Westwood
 DEA.L. adpcm_ms             ADPCM Microsoft
 D.A.L. adpcm_sbpro_2        ADPCM Sound Blaster Pro 2-bit
 D.A.L. adpcm_sbpro_3        ADPCM Sound Blaster Pro 2.6-bit
 D.A.L. adpcm_sbpro_4        ADPCM Sound Blaster Pro 4-bit
 DEA.L. adpcm_swf            ADPCM Shockwave Flash
 D.A.L. adpcm_thp            ADPCM Nintendo Gamecube THP
 D.A.L. adpcm_xa             ADPCM CDROM XA
 DEA.L. adpcm_yamaha         ADPCM Yamaha
 DEA..S alac                 ALAC (Apple Lossless Audio Codec)
 DEA.L. amr_nb               AMR-NB (Adaptive Multi-Rate NarrowBand) (decoders: amrnb libopencore_amrnb ) (encoders: libopencore_amrnb )
 D.A.L. amr_wb               AMR-WB (Adaptive Multi-Rate WideBand) (decoders: amrwb libopencore_amrwb )
 D.A..S ape                  Monkey's Audio
 D.A.L. atrac1               Atrac 1 (Adaptive TRansform Acoustic Coding)
 D.A.L. atrac3               Atrac 3 (Adaptive TRansform Acoustic Coding 3)
 ..A.L. atrac3p              Sony ATRAC3+
 D.A.L. binkaudio_dct        Bink Audio (DCT)
 D.A.L. binkaudio_rdft       Bink Audio (RDFT)
 D.A.L. bmv_audio            Discworld II BMV audio
 ..A.L. celt                 Constrained Energy Lapped Transform (CELT)
 DEA.L. comfortnoise         RFC 3389 Comfort Noise
 D.A.L. cook                 Cook / Cooker / Gecko (RealAudio G2)
 D.A.L. dsicinaudio          Delphine Software International CIN audio
 DEA.LS dts                  DCA (DTS Coherent Acoustics) (decoders: dca ) (encoders: dca )
 ..A.L. dvaudio              
 DEA.L. eac3                 ATSC A/52B (AC-3, E-AC-3)
 D.A.L. evrc                 EVRC (Enhanced Variable Rate Codec)
 DEA..S flac                 FLAC (Free Lossless Audio Codec)
 DEA.L. g723_1               G.723.1
 D.A.L. g729                 G.729
 DEA.L. gsm                  GSM (decoders: gsm libgsm ) (encoders: libgsm )
 DEA.L. gsm_ms               GSM Microsoft variant (decoders: gsm_ms libgsm_ms ) (encoders: libgsm_ms )
 D.A.L. iac                  IAC (Indeo Audio Coder)
 ..A.L. ilbc                 iLBC (Internet Low Bitrate Codec)
 D.A.L. imc                  IMC (Intel Music Coder)
 D.A.L. interplay_dpcm       DPCM Interplay
 D.A.L. mace3                MACE (Macintosh Audio Compression/Expansion) 3:1
 D.A.L. mace6                MACE (Macintosh Audio Compression/Expansion) 6:1
 D.A..S mlp                  MLP (Meridian Lossless Packing)
 D.A.L. mp1                  MP1 (MPEG audio layer 1) (decoders: mp1 mp1float )
 DEA.L. mp2                  MP2 (MPEG audio layer 2) (decoders: mp2 mp2float ) (encoders: mp2 libtwolame )
 DEA.L. mp3                  MP3 (MPEG audio layer 3) (decoders: mp3 mp3float ) (encoders: libmp3lame )
 D.A.L. mp3adu               ADU (Application Data Unit) MP3 (MPEG audio layer 3) (decoders: mp3adu mp3adufloat )
 D.A.L. mp3on4               MP3onMP4 (decoders: mp3on4 mp3on4float )
 D.A..S mp4als               MPEG-4 Audio Lossless Coding (ALS) (decoders: als )
 D.A.L. musepack7            Musepack SV7 (decoders: mpc7 )
 D.A.L. musepack8            Musepack SV8 (decoders: mpc8 )
 DEA.L. nellymoser           Nellymoser Asao
 DEA.L. opus                 Opus (Opus Interactive Audio Codec) (decoders: libopus ) (encoders: libopus )
 D.A.L. paf_audio            Amazing Studio Packed Animation File Audio
 DEA.L. pcm_alaw             PCM A-law / G.711 A-law
 D.A..S pcm_bluray           PCM signed 16|20|24-bit big-endian for Blu-ray media
 D.A..S pcm_dvd              PCM signed 20|24-bit big-endian
 DEA..S pcm_f32be            PCM 32-bit floating point big-endian
 DEA..S pcm_f32le            PCM 32-bit floating point little-endian
 DEA..S pcm_f64be            PCM 64-bit floating point big-endian
 DEA..S pcm_f64le            PCM 64-bit floating point little-endian
 D.A..S pcm_lxf              PCM signed 20-bit little-endian planar
 DEA.L. pcm_mulaw            PCM mu-law / G.711 mu-law
 DEA..S pcm_s16be            PCM signed 16-bit big-endian
 DEA..S pcm_s16be_planar     PCM signed 16-bit big-endian planar
 DEA..S pcm_s16le            PCM signed 16-bit little-endian
 DEA..S pcm_s16le_planar     PCM signed 16-bit little-endian planar
 DEA..S pcm_s24be            PCM signed 24-bit big-endian
 DEA..S pcm_s24daud          PCM D-Cinema audio signed 24-bit
 DEA..S pcm_s24le            PCM signed 24-bit little-endian
 DEA..S pcm_s24le_planar     PCM signed 24-bit little-endian planar
 DEA..S pcm_s32be            PCM signed 32-bit big-endian
 DEA..S pcm_s32le            PCM signed 32-bit little-endian
 DEA..S pcm_s32le_planar     PCM signed 32-bit little-endian planar
 DEA..S pcm_s8               PCM signed 8-bit
 DEA..S pcm_s8_planar        PCM signed 8-bit planar
 DEA..S pcm_u16be            PCM unsigned 16-bit big-endian
 DEA..S pcm_u16le            PCM unsigned 16-bit little-endian
 DEA..S pcm_u24be            PCM unsigned 24-bit big-endian
 DEA..S pcm_u24le            PCM unsigned 24-bit little-endian
 DEA..S pcm_u32be            PCM unsigned 32-bit big-endian
 DEA..S pcm_u32le            PCM unsigned 32-bit little-endian
 DEA..S pcm_u8               PCM unsigned 8-bit
 D.A.L. pcm_zork             PCM Zork
 D.A.L. qcelp                QCELP / PureVoice
 D.A.L. qdm2                 QDesign Music Codec 2
 ..A.L. qdmc                 QDesign Music
 DEA.L. ra_144               RealAudio 1.0 (14.4K) (decoders: real_144 ) (encoders: real_144 )
 D.A.L. ra_288               RealAudio 2.0 (28.8K) (decoders: real_288 )
 D.A..S ralf                 RealAudio Lossless
 DEA.L. roq_dpcm             DPCM id RoQ
 DEA..S s302m                SMPTE 302M
 D.A..S shorten              Shorten
 D.A.L. sipr                 RealAudio SIPR / ACELP.NET
 D.A.L. smackaudio           Smacker audio (decoders: smackaud )
 ..A.L. smv                  SMV (Selectable Mode Vocoder)
 D.A.L. sol_dpcm             DPCM Sol
 DEA... sonic                Sonic
 .EA... sonicls              Sonic lossless
 DEA.L. speex                Speex (decoders: libspeex ) (encoders: libspeex )
 D.A..S tak                  TAK (Tom's lossless Audio Kompressor)
 D.A..S truehd               TrueHD
 D.A.L. truespeech           DSP Group TrueSpeech
 DEA..S tta                  TTA (True Audio)
 D.A.L. twinvq               VQF TwinVQ
 D.A.L. vima                 LucasArts VIMA audio
 D.A.L. vmdaudio             Sierra VMD audio
 DEA.L. vorbis               Vorbis (decoders: vorbis libvorbis ) (encoders: vorbis libvorbis )
 ..A.L. voxware              Voxware RT29 Metasound
 D.A... wavesynth            Wave synthesis pseudo-codec
 D.A.LS wavpack              WavPack
 D.A.L. westwood_snd1        Westwood Audio (SND1) (decoders: ws_snd1 )
 D.A..S wmalossless          Windows Media Audio Lossless
 D.A.L. wmapro               Windows Media Audio 9 Professional
 DEA.L. wmav1                Windows Media Audio 1
 DEA.L. wmav2                Windows Media Audio 2
 D.A.L. wmavoice             Windows Media Audio Voice
 D.A.L. xan_dpcm             DPCM Xan
 ..D... dvd_nav_packet       DVD Nav packet
 ..D... klv                  SMPTE 336M Key-Length-Value (KLV) metadata
 DES... ass                  ASS (Advanced SSA) subtitle
 DES... dvb_subtitle         DVB subtitles (decoders: dvbsub ) (encoders: dvbsub )
 ..S... dvb_teletext         DVB teletext
 DES... dvd_subtitle         DVD subtitles (decoders: dvdsub ) (encoders: dvdsub )
 ..S... eia_608              EIA-608 closed captions
 D.S... hdmv_pgs_subtitle    HDMV Presentation Graphic Stream subtitles (decoders: pgssub )
 D.S... jacosub              JACOsub subtitle
 D.S... microdvd             MicroDVD subtitle
 DES... mov_text             MOV text
 D.S... mpl2                 MPL2 subtitle
 D.S... pjs                  PJS (Phoenix Japanimation Society) subtitle
 D.S... realtext             RealText subtitle
 D.S... sami                 SAMI subtitle
 DES... srt                  SubRip subtitle with embedded timing
 DES... ssa                  SSA (SubStation Alpha) subtitle
 DES... subrip               SubRip subtitle
 D.S... subviewer            SubViewer subtitle
 D.S... subviewer1           SubViewer v1 subtitle
 D.S... text                 raw UTF-8 text
 D.S... vplayer              VPlayer subtitle
 D.S... webvtt               WebVTT subtitle
 DES... xsub                 XSUB

all ffmpeg codecs that are supported

Codecs:
 D..... = Decoding supported
 .E.... = Encoding supported
 ..V... = Video codec
 ..A... = Audio codec
 ..S... = Subtitle codec
 ...S.. = Supports draw_horiz_band
 ....D. = Supports direct rendering method 1
 .....T = Supports weird frame truncation
 ------
 D V D  4xm             4X Movie
 D V D  8bps            QuickTime 8BPS video
 D A    8svx_exp        8SVX exponential
 D A    8svx_fib        8SVX fibonacci
 D A    8svx_raw        8SVX rawaudio
  EV    a64multi        Multicolor charset for Commodore 64
  EV    a64multi5       Multicolor charset for Commodore 64, extended with 5th color (colram)
 DEA    aac             Advanced Audio Coding
 D A    aac_latm        AAC LATM (Advanced Audio Codec LATM syntax)
 D V D  aasc            Autodesk RLE
 DEA    ac3             ATSC A/52A (AC-3)
  EA    ac3_fixed       ATSC A/52A (AC-3)
 D A    adpcm_4xm       ADPCM 4X Movie
 DEA    adpcm_adx       SEGA CRI ADX ADPCM
 D A    adpcm_ct        ADPCM Creative Technology
 D A    adpcm_ea        ADPCM Electronic Arts
 D A    adpcm_ea_maxis_xa ADPCM Electronic Arts Maxis CDROM XA
 D A    adpcm_ea_r1     ADPCM Electronic Arts R1
 D A    adpcm_ea_r2     ADPCM Electronic Arts R2
 D A    adpcm_ea_r3     ADPCM Electronic Arts R3
 D A    adpcm_ea_xas    ADPCM Electronic Arts XAS
 D A    adpcm_ima_amv   ADPCM IMA AMV
 D A    adpcm_ima_dk3   ADPCM IMA Duck DK3
 D A    adpcm_ima_dk4   ADPCM IMA Duck DK4
 D A    adpcm_ima_ea_eacs ADPCM IMA Electronic Arts EACS
 D A    adpcm_ima_ea_sead ADPCM IMA Electronic Arts SEAD
 D A    adpcm_ima_iss   ADPCM IMA Funcom ISS
 DEA    adpcm_ima_qt    ADPCM IMA QuickTime
 D A    adpcm_ima_smjpeg ADPCM IMA Loki SDL MJPEG
 DEA    adpcm_ima_wav   ADPCM IMA WAV
 D A    adpcm_ima_ws    ADPCM IMA Westwood
 DEA    adpcm_ms        ADPCM Microsoft
 D A    adpcm_sbpro_2   ADPCM Sound Blaster Pro 2-bit
 D A    adpcm_sbpro_3   ADPCM Sound Blaster Pro 2.6-bit
 D A    adpcm_sbpro_4   ADPCM Sound Blaster Pro 4-bit
 DEA    adpcm_swf       ADPCM Shockwave Flash
 D A    adpcm_thp       ADPCM Nintendo Gamecube THP
 D A    adpcm_xa        ADPCM CDROM XA
 DEA    adpcm_yamaha    ADPCM Yamaha
 DEA    alac            ALAC (Apple Lossless Audio Codec)
 D A    als             MPEG-4 Audio Lossless Coding (ALS)
 D A    amrnb           Adaptive Multi-Rate NarrowBand
 D A    amrwb           Adaptive Multi-Rate WideBand
 D V    amv             AMV Video
 D V D  anm             Deluxe Paint Animation
 D V D  ansi            ASCII/ANSI art
 D A    ape             Monkey's Audio
 DES    ass             Advanced SubStation Alpha subtitle
 DEV D  asv1            ASUS V1
 DEV D  asv2            ASUS V2
 D A    atrac1          Atrac 1 (Adaptive TRansform Acoustic Coding)
 D A    atrac3          Atrac 3 (Adaptive TRansform Acoustic Coding 3)
 D V D  aura            Auravision AURA
 D V D  aura2           Auravision Aura 2
 D V D  avs             AVS (Audio Video Standard) video
 D V D  bethsoftvid     Bethesda VID video
 D V D  bfi             Brute Force & Ignorance
 D A    binkaudio_dct   Bink Audio (DCT)
 D A    binkaudio_rdft  Bink Audio (RDFT)
 D V    binkvideo       Bink video
 DEV D  bmp             BMP image
 D V D  c93             Interplay C93
 D V D  camstudio       CamStudio
 D V D  camtasia        TechSmith Screen Capture Codec
 D V D  cavs            Chinese AVS video (AVS1-P2, JiZhun profile)
 D V D  cdgraphics      CD Graphics video
 D V D  cinepak         Cinepak
 D V D  cljr            Cirrus Logic AccuPak
 D A    cook            COOK
 D V D  cyuv            Creative YUV (CYUV)
 DEA    dca             
 D V D  dfa             Chronomaster DFA
 DEV D  dnxhd           VC3/DNxHD
 DEV    dpx             DPX image
 D A    dsicinaudio     Delphine Software International CIN audio
 D V D  dsicinvideo     Delphine Software International CIN video
 DES    dvbsub          DVB subtitles
 DES    dvdsub          DVD subtitles
 DEV D  dvvideo         DV (Digital Video)
 D V D  dxa             Feeble Files/ScummVM DXA
 DEA    eac3            ATSC A/52 E-AC-3
 D V D  eacmv           Electronic Arts CMV video
 D V D  eamad           Electronic Arts Madcow Video
 D V D  eatgq           Electronic Arts TGQ video
 D V    eatgv           Electronic Arts TGV video
 D V D  eatqi           Electronic Arts TQI Video
 D V D  escape124       Escape 124
 DEV D  ffv1            FFmpeg video codec #1
 DEVSD  ffvhuff         Huffyuv FFmpeg variant
 DEA    flac            FLAC (Free Lossless Audio Codec)
 DEV D  flashsv         Flash Screen Video
  EV    flashsv2        Flash Screen Video Version 2
 D V D  flic            Autodesk Animator Flic video
 DEVSD  flv             Flash Video (FLV) / Sorenson Spark / Sorenson H.263
 D V D  fraps           Fraps
 D V D  frwu            Forward Uncompressed
 DEA    g722            G.722 ADPCM
 DEA    g726            G.726 ADPCM
 DEV D  gif             GIF (Graphics Interchange Format)
 D A    gsm             GSM
 D A    gsm_ms          GSM Microsoft variant
 DEV D  h261            H.261
 DEVSDT h263            H.263 / H.263-1996
 D VSD  h263i           Intel H.263
  EV    h263p           H.263+ / H.263-1998 / H.263 version 2
 D V D  h264            H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
 D V D  h264_vdpau      H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (VDPAU acceleration)
 DEVSD  huffyuv         Huffyuv / HuffYUV
 D V D  idcinvideo      id Quake II CIN video
 D V D  iff_byterun1    IFF ByteRun1
 D V D  iff_ilbm        IFF ILBM
 D A    imc             IMC (Intel Music Coder)
 D V D  indeo2          Intel Indeo 2
 D V D  indeo3          Intel Indeo 3
 D V    indeo5          Intel Indeo Video Interactive 5
 D A    interplay_dpcm  DPCM Interplay
 D V D  interplayvideo  Interplay MVE video
 D V    j2k             
 DEV D  jpegls          JPEG-LS
 D V D  jv              Bitmap Brothers JV video
 D V    kgv1            Kega Game Video
 D V D  kmvc            Karl Morton's video codec
 D V D  lagarith        Lagarith lossless
  EV    libdirac        libdirac Dirac 2.2
 DEA    libgsm          libgsm GSM
 DEA    libgsm_ms       libgsm GSM Microsoft variant
  EA    libmp3lame      libmp3lame MP3 (MPEG audio layer 3)
 DEA    libopencore_amrnb OpenCORE Adaptive Multi-Rate (AMR) Narrow-Band
 D A    libopencore_amrwb OpenCORE Adaptive Multi-Rate (AMR) Wide-Band
 DEV    libschroedinger libschroedinger Dirac 2.2
 DEA    libspeex        libspeex Speex Encoder
  EV    libtheora       libtheora Theora
  EA    libvorbis       libvorbis Vorbis
 DEV    libvpx          libvpx VP8
  EV    libx264         libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
  EV    libxvid         libxvidcore MPEG-4 part 2
  EV    ljpeg           Lossless JPEG
 D V D  loco            LOCO
 D A    mace3           MACE (Macintosh Audio Compression/Expansion) 3:1
 D A    mace6           MACE (Macintosh Audio Compression/Expansion) 6:1
 D V D  mdec            Sony PlayStation MDEC (Motion DECoder)
 D V D  mimic           Mimic
 DEV D  mjpeg           MJPEG (Motion JPEG)
 D V D  mjpegb          Apple MJPEG-B
 D A    mlp             MLP (Meridian Lossless Packing)
 D V D  mmvideo         American Laser Games MM Video
 D V D  motionpixels    Motion Pixels video
 D A    mp1             MP1 (MPEG audio layer 1)
 D A    mp1float        MP1 (MPEG audio layer 1)
 DEA    mp2             MP2 (MPEG audio layer 2)
 D A    mp2float        MP2 (MPEG audio layer 2)
 D A    mp3             MP3 (MPEG audio layer 3)
 D A    mp3adu          ADU (Application Data Unit) MP3 (MPEG audio layer 3)
 D A    mp3adufloat     ADU (Application Data Unit) MP3 (MPEG audio layer 3)
 D A    mp3float        MP3 (MPEG audio layer 3)
 D A    mp3on4          MP3onMP4
 D A    mp3on4float     MP3onMP4
 D A    mpc7            Musepack SV7
 D A    mpc8            Musepack SV8
 DEVSDT mpeg1video      MPEG-1 video
 D V DT mpeg1video_vdpau MPEG-1 video (VDPAU acceleration)
 DEVSDT mpeg2video      MPEG-2 video
 DEVSDT mpeg4           MPEG-4 part 2
 D V DT mpeg4_vdpau     MPEG-4 part 2 (VDPAU)
 D VSDT mpegvideo       MPEG-1 video
 D V DT mpegvideo_vdpau MPEG-1/2 video (VDPAU acceleration)
 D VSDT mpegvideo_xvmc  MPEG-1/2 video XvMC (X-Video Motion Compensation)
 DEVSD  msmpeg4         MPEG-4 part 2 Microsoft variant version 3
 D VSD  msmpeg4v1       MPEG-4 part 2 Microsoft variant version 1
 DEVSD  msmpeg4v2       MPEG-4 part 2 Microsoft variant version 2
 D V D  msrle           Microsoft RLE
 DEV D  msvideo1        Microsoft Video-1
 D V D  mszh            LCL (LossLess Codec Library) MSZH
 D V D  mxpeg           Mobotix MxPEG video
 DEA    nellymoser      Nellymoser Asao
 D V D  nuv             NuppelVideo/RTJPEG
 DEV D  pam             PAM (Portable AnyMap) image
 DEV D  pbm             PBM (Portable BitMap) image
 DEA    pcm_alaw        PCM A-law
 D A    pcm_bluray      PCM signed 16|20|24-bit big-endian for Blu-ray media
 D A    pcm_dvd         PCM signed 20|24-bit big-endian
 DEA    pcm_f32be       PCM 32-bit floating point big-endian
 DEA    pcm_f32le       PCM 32-bit floating point little-endian
 DEA    pcm_f64be       PCM 64-bit floating point big-endian
 DEA    pcm_f64le       PCM 64-bit floating point little-endian
 D A    pcm_lxf         PCM signed 20-bit little-endian planar
 DEA    pcm_mulaw       PCM mu-law
 DEA    pcm_s16be       PCM signed 16-bit big-endian
 DEA    pcm_s16le       PCM signed 16-bit little-endian
 D A    pcm_s16le_planar PCM 16-bit little-endian planar
 DEA    pcm_s24be       PCM signed 24-bit big-endian
 DEA    pcm_s24daud     PCM D-Cinema audio signed 24-bit
 DEA    pcm_s24le       PCM signed 24-bit little-endian
 DEA    pcm_s32be       PCM signed 32-bit big-endian
 DEA    pcm_s32le       PCM signed 32-bit little-endian
 DEA    pcm_s8          PCM signed 8-bit
 DEA    pcm_u16be       PCM unsigned 16-bit big-endian
 DEA    pcm_u16le       PCM unsigned 16-bit little-endian
 DEA    pcm_u24be       PCM unsigned 24-bit big-endian
 DEA    pcm_u24le       PCM unsigned 24-bit little-endian
 DEA    pcm_u32be       PCM unsigned 32-bit big-endian
 DEA    pcm_u32le       PCM unsigned 32-bit little-endian
 DEA    pcm_u8          PCM unsigned 8-bit
 DEA    pcm_zork        PCM Zork
 DEV D  pcx             PC Paintbrush PCX image
 DEV D  pgm             PGM (Portable GrayMap) image
 DEV D  pgmyuv          PGMYUV (Portable GrayMap YUV) image
 D S    pgssub          HDMV Presentation Graphic Stream subtitles
 D V D  pictor          Pictor/PC Paint
 DEV D  png             PNG image
 DEV D  ppm             PPM (Portable PixelMap) image
 D V D  ptx             V.Flash PTX image
 D A    qcelp           QCELP / PureVoice
 D A    qdm2            QDesign Music Codec 2
 D V D  qdraw           Apple QuickDraw
 D V D  qpeg            Q-team QPEG
 DEV D  qtrle           QuickTime Animation (RLE) video
 D V D  r10k            AJA Kona 10-bit RGB Codec
 D V D  r210            Uncompressed RGB 10-bit
 DEV    rawvideo        raw video
 DEA    real_144        RealAudio 1.0 (14.4K) encoder
 D A    real_288        RealAudio 2.0 (28.8K)
 D V D  rl2             RL2 video
 DEA    roq_dpcm        id RoQ DPCM
 DEV D  roqvideo        id RoQ video
 D V D  rpza            QuickTime video (RPZA)
 DEV D  rv10            RealVideo 1.0
 DEV D  rv20            RealVideo 2.0
 D V D  rv30            RealVideo 3.0
 D V D  rv40            RealVideo 4.0
 D A    s302m           SMPTE 302M
 DEV    sgi             SGI image
 D A    shorten         Shorten
 D A    sipr            RealAudio SIPR / ACELP.NET
 D A    smackaud        Smacker audio
 D V D  smackvid        Smacker video
 D V D  smc             QuickTime Graphics (SMC)
 DEV D  snow            Snow
 D A    sol_dpcm        DPCM Sol
 DEA    sonic           Sonic
  EA    sonicls         Sonic lossless
 D V D  sp5x            Sunplus JPEG (SP5X)
 DES    srt             SubRip subtitle
 D V D  sunrast         Sun Rasterfile image
 DEV D  svq1            Sorenson Vector Quantizer 1 / Sorenson Video 1 / SVQ1
 D VSD  svq3            Sorenson Vector Quantizer 3 / Sorenson Video 3 / SVQ3
 DEV D  targa           Truevision Targa image
 D VSD  theora          Theora
 D V D  thp             Nintendo Gamecube THP video
 D V D  tiertexseqvideo Tiertex Limited SEQ video
 DEV D  tiff            TIFF image
 D V D  tmv             8088flex TMV
 D A    truehd          TrueHD
 D V D  truemotion1     Duck TrueMotion 1.0
 D V D  truemotion2     Duck TrueMotion 2.0
 D A    truespeech      DSP Group TrueSpeech
 D A    tta             True Audio (TTA)
 D A    twinvq          VQF TwinVQ
 D V D  txd             Renderware TXD (TeXture Dictionary) image
 D V D  ultimotion      IBM UltiMotion
 DEV D  v210            Uncompressed 4:2:2 10-bit
 D V D  v210x           Uncompressed 4:2:2 10-bit
 D V    vb              Beam Software VB
 D V D  vc1             SMPTE VC-1
 D V D  vc1_vdpau       SMPTE VC-1 VDPAU
 D V D  vcr1            ATI VCR1
 D A    vmdaudio        Sierra VMD audio
 D V D  vmdvideo        Sierra VMD video
 D V D  vmnc            VMware Screen Codec / VMware Video
 DEA    vorbis          Vorbis
 D VSD  vp3             On2 VP3
 D V D  vp5             On2 VP5
 D V D  vp6             On2 VP6
 D V D  vp6a            On2 VP6 (Flash version, with alpha channel)
 D V D  vp6f            On2 VP6 (Flash version)
 D V D  vp8             On2 VP8
 D V D  vqavideo        Westwood Studios VQA (Vector Quantized Animation) video
 D A    wavpack         WavPack
 D A    wmapro          Windows Media Audio 9 Professional
 DEA    wmav1           Windows Media Audio 1
 DEA    wmav2           Windows Media Audio 2
 D A    wmavoice        Windows Media Audio Voice
 DEVSD  wmv1            Windows Media Video 7
 DEVSD  wmv2            Windows Media Video 8
 D V D  wmv3            Windows Media Video 9
 D V D  wmv3_vdpau      Windows Media Video 9 VDPAU
 D V D  wnv1            Winnov WNV1
 D A    ws_snd1         Westwood Audio (SND1)
 D A    xan_dpcm        DPCM Xan
 D V D  xan_wc3         Wing Commander III / Xan
 D V D  xan_wc4         Wing Commander IV / Xxan
 D V D  xl              Miro VideoXL
 DES    xsub            DivX subtitles (XSUB)
 D V    yop             Psygnosis YOP Video
 DEV D  zlib            LCL (LossLess Codec Library) ZLIB
 DEV D  zmbv            Zip Motion Blocks Video

Note, the names of encoders and decoders do not always match, so there are
several cases where the above table shows encoder only or decoder only entries
even though both encoding and decoding are supported. For example, the h263
decoder corresponds to the h263 and h263p encoders, for file formats it is even
worse.

Download videos from YT channel

# youtube-dl -f best -ciw -o "%(title)s.%(ext)s" -v https://www.youtube.com/channel/UCUAVzSGkkTa-IoMx_x1XK4Q

Cool effects using ffplay

$ ffplay -i /dev/video0 -vf 'lagfun=decay=0.98[tmp]; [tmp] hue=180*sin(t)'

Using ffmpeg with hardware acceleration

$ ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i bulkerasenew.mp4 -c:v h264_vaapi -b:v 2M -maxrate 2M output.mp4

Multimedia

MythTV

Multimedia

Video (Kino, dvgrab, OpenShot, KdeNlive)

Multimedia

Dvgrab

To capture video from a firewire device use:

dvgrab -frames 0 -size 0 -format raw -noavc -showstatus -csize 1000 tape
dvgrab -frames 0 -size 0 -format dv2 -noavc -showstatus -csize 1000 tape
dvgrab -size 0 -format

best options:

dvgrab -autosplit -frames 0 -size 0 -format raw -noavc -showstatus -timestamp tape

where

-frames number    max number of frames per split
                         0 = unlimited [default 0]
-size number             max file size, 0 = unlimited [default 1000]
-csize number     split file when collections of files are about to exceed
                         number MiB, 0 = unlimited [default 0]
-format type      save as one of the following file types [default raw]
             raw         raw DV file with a .dv extension
             dif         raw DV file with a .dif extension
             dv1         'Type 1' DV AVI file
             dv2, avi    'Type 2' DV AVI file
             qt, mov     QuickTime DV movie
             mpeg2, hdv  MPEG-2 transport stream (HDV)
             jpeg, jpg   sequence of JPEG files (DV only)
-noavc                   disable use of AV/C VTR control
-srt                     generate .srt subtitle files containing the recording date and time in SRT format.
-t, -timestamp           Put information on date and time of recording into file name.
-a, -autosplit[=num]     Try to detect whenever a new recording starts, and store it into a separate file. This can be combined with the -frames and -size options, and a  split  occurs  whenever  a  specified  event arises.  Autosplit is off by default.

tape is the filename

Format types:


raw
Input #0, dv, from 'test-raw001.dv':                                                                                                                                                                                 
  Duration: 00:00:29.12, start: 0.000000, bitrate: 28800 kb/s
    Stream #0.0: Video: dvvideo, yuv420p, 720x576, 28800 kb/s, PAR 16:15 DAR 4:3, 25 tbr, 25 tbn, 25 tbc
    Stream #0.1: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s

dv1
Input #0, avi, from 'test-dv1001.avi':                                                                                                                                                                               
  Duration: 00:00:29.08, start: 0.000000, bitrate: 30336 kb/s
    Stream #0.0: Video: dvvideo, yuv420p, 720x576, 28800 kb/s, PAR 16:15 DAR 4:3, 25 tbr, 25 tbn, 25 tbc
    Stream #0.1: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s

dv2
Input #0, avi, from 'test-dv2001.avi':
  Duration: 00:00:27.64, start: 0.000000, bitrate: 30345 kb/s
    Stream #0.0: Video: dvvideo, yuv420p, 720x576 [PAR 16:15 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
    Stream #0.1: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s

Multimedia

Starting chromium as default browser in thunderbird

Thunderbird seems to always start firefox when a link is clicked. This is bad behaviour.

The following command will start chromium:

gconftool-2 --type string -s /desktop/gnome/url-handlers/http/command "/usr/bin/chromium %s"

This works also in KDE

Kdenlive

If you're spending your weeks in the timeline, you'll want to know a more efficient way of navigating it. There are all the usual video editing conventions in Kdenlive, albeit with some variation in keyboard assignments.

jkl

The now classic and ubiquitous "jkl" keyboard sequence is also present in Kdenlive. To play your timeline, you may press "l" (lowercase L) once for standard speed, "ll" for double speed, "lll" for triple speed. "K" stops playback. Press "j" for reverse playback, "jj" for double speed reverse, and "jjj" for triple speed reverse.

move by frame

You can also move by frame or by second. The left or right arrow will advance or rewind by one frame; shift-leftArrow or shift-rightArrow will move forward or back by one second.

jump

To move your playhead to the beginning of a video region, use the Home key; for the end of the region, use End.

To move along the timeline by splices, use Alt-leftArrow or Alt-rightArrow. And finally, you can jump to the beginning or end of the Timeline with control-Home or control-End.

By using these navigational tools, you'll find that for very long stretches of video editing, you won't need to bother with your mouse, which, as any Unix guru knows, is the real key to efficiency.

edit

cut: shift-r

Multimedia

Screenresolutions, dimensions and DPI

General tips

The ultimate solution to ugly fonts is to use more pixels to draw them. That translates to using a larger nominal pixel size. when that has the effect of making fonts too big, more screen resolution is required at the same physical size. More pixels has a cubic effect, meaning it doesn’t require much more in nominal size to have a big effect.

Sometimes in order to make all screen objects bigger, people use a lower screen resolution than the optimal one a screen supports. This has the opposite effect, making fonts spindly or otherwise ugly. IOW, if your screen supports 1920x1080 but you are using 1366x768, a switch to using 1920x1080 and an increase nominal font sizes in settings is required.

I keep screen DPI set to 120 on a 24" 1920x1200 screen. The combination of 24" and 1920x1200 calculates to a DPI of 94. The application of a logical DPI of 120 results in fonts big enough to see that are never ugly. A 12pt font @120 DPI uses ~200px per glyph, compared to ~128 used @96 DPI, >56% more at the same physical size. The screen can’t actually produce that much resolution, but the effect of trying by the X server, with or without subpixel, hinting or anti-aliasing, works like magic.

Envy

pvdm@Envy:~> xrandr
Screen 0: minimum 320 x 200, current 1920 x 1080, maximum 16384 x 16384
eDP-1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 344mm x 194mm
   1920x1080     60.00*+  40.00  
DP-1 disconnected (normal left inverted right x axis y axis)
DP-2 disconnected (normal left inverted right x axis y axis)
HDMI-1 disconnected (normal left inverted right x axis y axis)

DPI=140,093023256

142

lesie

sietske@lesie:~> xrandr
Screen 0: minimum 320 x 200, current 1920 x 1080, maximum 8192 x 8192
eDP-1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 344mm x 194mm
   1920x1080     60.03*+  60.01    59.97    59.96    59.93  

DISPLAY: 15.6”, Full HD (1920 x 1080), IPS
Diagonal    15.6"
Pixel density   141.21 ppi
Panel Type  IPS
Resolution  1920 x 1080
Becomes "Retina" from   24 inches (62 cm)

HP elitebook 8570p

Horizontal pixels: 1600
Vertical pixels: 900
Diagonal size (inches):  15,77
        
 
Pixels per Inch     PPI:     116.41  
Dot Pitch:   0.22    
Screen Width:    13.74   
Screen Height:   7.73


pvdm@lapx:~> xrandr
Screen 0: minimum 320 x 200, current 1600 x 900, maximum 16384 x 16384
LVDS connected primary 1600x900+0+0 (normal left inverted right x axis y axis) 345mm x 194mm
   1600x900       60.0*+   40.0  
   1440x900       59.9  
   1280x854       59.9  
   1280x800       59.8  
   1280x720       59.9  
   1152x768       59.8  
   1024x768       59.9  
   800x600        59.9  
   848x480        59.7  
   720x480        59.7  
   640x480        59.4

samsung rv720 bla

samsie:~ # xrandr
Screen 0: minimum 320 x 200, current 1600 x 900, maximum 16384 x 16384
LVDS connected primary 1600x900+0+0 (normal left inverted right x axis y axis) 345mm x 194mm
   1600x900      60.03*+  40.02  
   1440x900      59.89  
   1280x854      59.89  
   1280x800      59.81  
   1280x720      59.86  
   1152x768      59.78  
   1024x768      59.92  
   800x600       59.86  
   848x480       59.66  
   720x480       59.71  
   640x480       59.38

116 DPI PvdM

tabel

resolutie	Dichtheid
7” 15:9 15,3x9,15
800x480	133
8,9” 16:9 19,55x11,45
1024x600	133
10,2” 16:9 22,3x13,05
1024x600	117
10,6” 15:9 23,1x13,8
1280x768	141
11,1” 16:9 24,9x13,8
1366x768	141
12” 4:3 24,7x18,5
1024x768	105
1400x1050	144
12,1” 16:10 26,1x16,3
1280x800	125
1440x900	140
13” 16:9 29x16,35
1366x768	120
1600x900	140
13.1” 16:10 28,5x17,9
1280x800	114
1440x900	128
14” 4:3 28,5x21,4
1024x768	91
1400x1050	125
14,1” 16:10 30,4x19
1280x800	107
1440x900	120
15” 4:3 30,4x22,8
1024x768	86
1400x1050	117
1600x1200	134
15,4” 16:10 33,15x20,82
1280x800	98
1440x900	110
1680x1050	129
1920x1200	147
16” 16:9 35,3x19,9
1366x768	98
1920x1080	138
16,4” 16:9 36,3x20,4
1600x900	112
1920x1080	134
17,1” 16:10 36,8x23
1440x900	98
1680x1050	116
1920x1200	133
18,4” 16:9 40,9x23
1680x945	104
1920x1080	119
19” 16:10 41x25,6
1440x900	89
20” 16:10 43,4x27,1
1680x1050	98

Multimedia

VirtualBox

command line tips

Multimedia

Chromecast op linux

firewall

Allow Multicast in Firewalld

sudo firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0 -m udp -p udp -m pkttype --pkt-type multicast -j ACCEPT

open ports

tcp 8008-8009
udp 32768-61000

Chromium

The native "Cast..." does not work until you enable this flag below. Once its enabled, it works perfect, no need for any extension.

chrome://flags/#load-media-router-component-extension

set to enabled.

also Cast Media Route Provider

tested on Version 60.0.3112.113 (openSUSE Build) (64-bit)

Command line

catt

$ ~/.local/bin/catt -d Kitchen cast -s ./Men.in.Black.2.2002.1080p.BluRay.H264.AAC-RARBG.srt ./Men.in.Black.2.2002.1080p.BluRay.H264.AAC-RARBG.mp4
$ ~/.local/bin/catt -d "Kitchen" cast "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

Multimedia

Vlc tips & tricks

Startup options:

--start-time=<integer> Start time
The stream will start at this position (in seconds).
--stop-time=<integer> Stop time
The stream will stop at this position (in seconds).
--run-time=<integer> Run time
The stream will run this duration (in seconds).

get around the problem by adding vlc://pause:9999 entries to playlist after the titles.

#EXTM3U
#EXTINF:350,Steely Dan - Babylon Sisters
/data/audiophile/!!AA meeting okt 2018/01 - Babylon Sisters.flac
vlc://pause:4
#EXTINF:461,Jeff Wayne - Forever Autumn
/data/audiophile/!!AA meeting okt 2018/04 Jeff Wayne - Forever Autumn.flac
vlc://pause:4
#EXTINF:311,Dire Straits - So Far Away
/data/audiophile/!!AA meeting okt 2018/01 - So Far Away.flac
vlc://pause:4

see: http://wiki.videolan.org/VLC_command-line_help

Multimedia

Chromium and chromecast

To resolve this issue you need to enable chrome://flags/#load-media-router-component-extension

Multimedia

Video output info

Depending on the age, old consoles (and computers) output a worse video signal (quality-wise) than what the internal components actually produce. This was needed to be able to connect to TVs at the time. For example, the oldest output format is RF, which was connected to the antenna of TVs.

While this is still fine for some people, others prefer the best possible signal, which means feeding the color and sync signals individually to the monitor. The quality ladder from worst to best is:

RF
composite video
S-Video
component/YPbPr
RGBS/RGBHV.

The higher the quality, the more individual wires you have. While RF mixes everything including sound into one signal (hence one wire), with RGBHV you have 5 video + 2 sound wires.

The best an unmodified C64 can output is S-Video (2 wires for video signal and 1 for mono sound), hence to get something even better you need a special modification board that then produces component or RGBS video signals (depending on the settings).

Of course, this is all very simplified, nowadays, for some old computers/consoles you even have digital output modifications available that you can connect e.g. via HDMI.

It is this one (open source):

https://github.com/c0pperdragon/C64-Video-Enhancement

You can buy pre-built ones e.g. here:

https://videogameperfection.com/products/commodore-component-video/

These ship with an older firmware (2.6) though, I had to update (2.10) mine to get it working with my older C64 version.

Multimedia

Chrome options

/usr/bin/chromium -high-dpi-support=1 -force-device-scale-factor=1 --enable-features=UseOzonePlatform --ozone-platform=wayland %U

/usr/bin/google-chrome-stable -high-dpi-support=1 -force-device-scale-factor=1 --enable-features=UseOzonePlatform --ozone-platform=wayland %U

Multimedia

Use yt-dlp the correct way for music files

Use:

-f bestaudio
-f ba --extract-audio --audio-format flac

for best audio quality.

-o "%(title)s.%(ext)s"

for correct naming of the files.

--add-metadata

for correct IDs.

--parse-metadata "playlist_index:%(track_number)s"

for parsing track numbers.

--embed-thumbnail

This instructs yt-dlp to embed the thumbnail image in the downloaded audio file.

You can use youtube URLS to download complete album in one go:

#yt-dlp -f bestaudio -o "%(title)s.%(ext)s" --add-metadata --parse-metadata "playlist_index:%(track_number)s" https://www.youtube.com/playlist?list=OLAK5uy_nBJgNLqUL8A64tQMKFNPK3x7KzNOgfbUw

Embed thumbnail in opus music files

#ffmpeg -i audio_file.opus -i thumbnail.jpg -c:v mjpeg -c:a copy -metadata:s:v title="Album Art" output_file.opus

Multimedia

Use yt-dlp with subtitles for video files

To download video with english subtitles:

# yt-dlp --write-sub --sub-lang "en.*" {URL}

This downloads all variants of the english language

Multimedia

Yt-dlp

Multimedia

Convert .img to .vdi for use with VirtualBox

VBoxManage convertfromraw -format VDI [filename].img [filename].vdi

Multimedia

Burning CD/DVD from CLI with wodim

List devices:

#wodim --devices

Check devices:

#wodim -checkdrive

Burn .ISO:

#wodim dev=/dev/scd0 -burnfree -v -data bla.iso

Multimedia

Convert an AVCHD / MTS file to MP4 usinf ffmpeg

REDIRECT Convert an AVCHD / MTS file to MP4 using ffmpeg

Multimedia

Embed video in html5

<video width="640" height="480">
   <source type="video/ogg" src="foo.ogg">
      <!-- and a fallback for h264 -->
      <source type="video/mp4" src="foo.mp4">
</video>

Netwerk & Server

MySQL

Netwerk & Server

Fixing CUPS login problems on SUSE

SUSE: What Password Do I Use At The Web Interface?

SuSE defaults to using Digest passwords and "RunAsUser yes", which means you have to use the lppasswd command to add accounts and passwords.

To add a user named Steve:

lppasswd -a Steve

Then enter and confirm a password.

SUSE does not follow the normal CUPS defaults and the lppasswd information doesn't follow changes to the normal accounts, causing a lot of confusion.

Any problems using this set up should be addressed to SUSE.

To set SUSE to the standard default settings of CUPS:

1) Goto /etc/cups/cups.conf

2) Change in the AuthType: AuthType Basic

3) Change RunAsUser to No.

Netwerk & Server

CUPS

Netwerk & Server

How to make your CUPS server broadcast itself

In order for your clients to automatically recognize the printers that CUPS has, add/edit the following setting in /etc/cups/cupsd.conf:

BrowseAddress @LOCAL

In this way, your clients won't have to do anything, as their (local) CUPS server will recognize the broadcasts from your (global) CUPS server!

Netwerk & Server

Install a PXE boot server on 11.1

This document shows quick and dirty how to install a working PXE boot server on openSUSE

We start of course with a working server with a static IP address and a working DHCP server somewhere (could be on your server or on your home router or something else). If you haven't got one already, please make sure that you get this working before you continue!

install the atftp and syslinux package s

#zypper in atftp syslinux

possibly create the /tftpboot/pxelinux.cfg dir

#mkdir /tftpboot/pxelinux.cfg

possibly open port in FireWall
copy the file /usr/share/syslinux/pxelinux.0 to /tftpboot

#cp /usr/share/syslinux/pxelinux.0 /tftpboot

create the file /tftpboot/pxelinux.cfg/default (#vi /tftpboot/pxelinux.cfg/default) with the following content:

default linux
prompt 1
timeout 30
label linux
kernel linux
append initrd=initrd splash=silent
showopts

or put file 'message' in /tftpboot with the following example content:

Welcome to Provisioning!
Choose one of the following options and press .
sles10.2       - SUSE Linux Enterprise Server SP2 32 bit
sles10.2_64    - SUSE Linux Enterprise Server SP2 64 bit
opensuse11_32  - OpenSUSE 11, 32 bit
opensuse11_64  - OpenSUSE 11, 64 bit
hardy          - Ubuntu 8.04 (Hardy Heron)
thin           - Boot thin client
next           - Boot from next BIOS device (default)
disk           - Boot from first BIOS disk

put 2 files initrd and linux in the /tftpboot dir

: 32 bit: from the /boot/i386/loader directory on the CD/DVD; 64 bit: from the /boot/x86_64/loader directory on the CD/DVD

That should do the trick!

Netwerk & Server

Apache en HTML

Apache

Netwerk & Server

How to connect to server on strange port, or when you're behind a firewall

Situation

cpuxxxx -> FW -> intermediateserver -> targetserver

22......-> 22 -> 22.................-> portnumber_on_targetserver

Connect

On cpuxxxx:

ssh -L portnumber_on_targetserver:targetserver:portnumber_on_targetserver username_on_intermediateserver@intermediateserver

Leave terminal open!

Then on cpuxxxx again:

ssh -YC username_on_targetserver@localhost -p portnumber_on_targetserver

That's it!!

Copy files

from here tot server:

scp -P portnumber_on_targetserver file user@localhost:/path/were/to/put/it/

from server to local:

scp -P portnumber_on_targetserver user@localhost:/path/were/to/get/it/from/ .

mount filesystem over ssh using sshfs

sshfs user@localhost:/data/videos/ mnt/ -p portnumber

Rsync

from server to localdisk:

rsync -avz -e 'ssh -p PORTNUMBER' USER@localhost:/data/htdocs/motion/ /data/motion/

Start Konversation from behind a firewall

ssh -L 6667:irc.freenode.org:6667 user@xs3.xs4all.nl
konversation --server localhost

Start Pidgin from home to wzs

in a terminal typ:

ssh -L 8300:gwmsg.wonen.amsterdam.nl:8300 root@wzs

leave terminal open
start pidgin and fill in:

: server: localhost; port: 8300

reverse tunnel

From the machine at work, create a tunnel to your machine running ssh at home:

randolf@workserver:~$ ssh -vvv randolf@myhomeserver -R 44455:localhost:22

This will forward the remote port 44455 from your home server to port 22 (or whichever port ssh is listening on) on your work machine.

From home, check to see if anything is listening on 44455

randolf@homeserver:~$ netstat -an | grep 44455
tcp 0 0 127.0.0.1:44455 0.0.0.0:* LISTEN

Next, to connect from home to your work machine, from your home server:

randolf@homeserver:~$ ssh localhost -p 44455
Password: ******

From there you should be connected to your work machine via your tunnel.

Netwerk & Server

Mounting remote directories using FUSE and sshfs on openSUSE

sshfs user@host: /mnt/

Once that’s done, the remote filesystem is mounted as the directory “rsync” and I can copy and move files just like they’re local.

Netwerk & Server

MediaWiki

Netwerk & Server

How to make a dump of a website

update 20210611 Best Version:

wget -m https://

original

If you want to make a dump of a website, then you can do:

/usr/bin/wget -q --convert-links -E -m http://website.nu.com.mp3

wget --progress=bar --convert-links -E -m ftp://ftp.studer.ch/Public/

Actually, to download a single page and all its requisites (even if they exist on separate websites), and make sure the lot displays properly locally, this author likes to use a few options in addition to -p:

wget -E -H -k -K -p http:///

-E adjust extention ???

-H also download from foreign hosts

-p download all page requisites

PvdM version:

wget -r -k -K -p http://

-r recursive

-k convert links

-K backup converted

Netwerk & Server

Iptables for dummies

Netfilter

Elke Linux distributie gebruikt Netfilter (in de kernel) voor firewalling. De beheertool daarvoor is 'iptables', via de commandline.

Hoe werkt het? Er zijn drie soorten verkeer: INPUT, OUTPUT en FORWARD. Dit worden ook wel 'chains' genoemd.

INPUT is alles op je interface binnenkomt
OUTPUT is alles wat je interface verlaat
FORWARD is als de computer als router is geconfigureerd, dus met meerdere netwerkkaarten

Om te beginnen, moeten we de firewall service stoppen. Op openSUSE is dat

rcSuSEfirewall2 stop

controle

Vervolgens beginnen we met een schone lei. Verwijder alle firewall regels

iptables -F

Controleer dit met

iptables -L -v

dichtzetten

Hierna staat je firewall uit en is leeg, dus we gaan hem snel dichtzetten. We gebruiken hiervoor 'policies'.

iptables -p INPUT DROP
iptables -p OUTPUT DROP
iptables -p FORWARD DROP

-p betekent hier policy, INPUT is de chain waarop hij werkt, en de 'target' is 'drop'.

targets

Er zijn de volgende targets:

DROP: alle pakketjes worden gedropt, er is geen terugmelding
REJECT: het pakketje wordt verworpen, er gaat wel een melding terug
ACCEPT: het pakketje wordt geaccepteerd
LOG: er wordt een melding in syslog geschreven over het binnenkomende pakketje

Nu is de computer potdicht, en er is dus niet mee te werken met communicatie naar buiten. We gaan hem zo inrichten dat er communicatie naar buiten mogelijk is.

loopback goedzetten

Maar eerst moet er de loopback interface (lo) goed geconfigureerd worden. Hoewel niet strikt noodzakelijk, maar dit zorgt ervoor dat een hoop zaken goed verlopen.

iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

-A betekent 'append', aan het eind. De volgorde van de regels is belangrijk! De eerste 'hit' wordt uitgevoerd en er wordt daarna niet meer verder gekeken.

-i is de interface

-j (jump) naar welke target

Hierna kan er gepinged worden naar de loopbackinterface:

PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.034 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.019 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.027 ms

uitgaand verkeer toestaan

We gaan nu alle uitgaande verkeer toestaan.

iptables -A OUTPUT -o eth0 -j ACCEPT

Dit zorgt voor uitgaande pakketjes, maar antwoorden hierop die binnenkomen, kunnen nog steeds niet de computer bereiken.

iptables -A INPUT -m state - state=ESTABLISHED,RELATED -j ACCEPT

creert een stateful filter.

Netwerk & Server

How to get a list of permissions of mysql users

How to Get a List of Permissions of MySQL Users

MySQL has advanced database access system. A database user can have access to the whole database, separate tables, or to separate columns of tables. Also there is a restriction for actions a user may perform with records. MySQL server uses several tables in a special database to organize such a complicated database access structure. The access policy is based on the values available in these tables.

The database that MySQL server uses to store internal data about users is called mysql by default. There are tables for storing information about users’ accounts in this database:

user contains a list of all users of the MySQL server and their permissions to access the database;
db contains a list of databases with a matching list of database users and their privileges for executing operations;
tables_priv contains a list of database tables (views) that a user has access to;
columns_priv contains a list of columns from the database tables (views) a user has access to;
procs_priv contains a list of database procedures (functions) a user has access to.
To get the list of users’ privileges concerning data access, the following queries may be executed:

* the list of global privileges:

SELECT

mu.host `Host`,

mu.user `User`,

REPLACE(RTRIM(CONCAT(

IF(mu.Select_priv = 'Y', 'Select ', ''),

IF(mu.Insert_priv = 'Y', 'Insert ', ''),

IF(mu.Update_priv = 'Y', 'Update ', ''),

IF(mu.Delete_priv = 'Y', 'Delete ', ''),

IF(mu.Create_priv = 'Y', 'Create ', ''),

IF(mu.Drop_priv = 'Y', 'Drop ', ''),

IF(mu.Reload_priv = 'Y', 'Reload ', ''),

IF(mu.Shutdown_priv = 'Y', 'Shutdown ', ''),

IF(mu.Process_priv = 'Y', 'Process ', ''),

IF(mu.File_priv = 'Y', 'File ', ''),

IF(mu.Grant_priv = 'Y', 'Grant ', ''),

IF(mu.References_priv = 'Y', 'References ', ''),

IF(mu.Index_priv = 'Y', 'Index ', ''),

IF(mu.Alter_priv = 'Y', 'Alter ', ''),

IF(mu.Show_db_priv = 'Y', 'Show_db ', ''),

IF(mu.Super_priv = 'Y', 'Super ', ''),

IF(mu.Create_tmp_table_priv = 'Y', 'Create_tmp_table ', ''),

IF(mu.Lock_tables_priv = 'Y', 'Lock_tables ', ''),

IF(mu.Execute_priv = 'Y', 'Execute ', ''),

IF(mu.Repl_slave_priv = 'Y', 'Repl_slave ', ''),

IF(mu.Repl_client_priv = 'Y', 'Repl_client ', ''),

IF(mu.Create_view_priv = 'Y', 'Create_view ', ''),

IF(mu.Show_view_priv = 'Y', 'Show_view ', ''),

IF(mu.Create_routine_priv = 'Y', 'Create_routine ', ''),

IF(mu.Alter_routine_priv = 'Y', 'Alter_routine ', ''),

IF(mu.Create_user_priv = 'Y', 'Create_user ', ''),

IF(mu.Event_priv = 'Y', 'Event ', ''),

IF(mu.Trigger_priv = 'Y', 'Trigger ', '')

)), ' ', ', ') AS `Privileges`

FROM

mysql.user mu

ORDER BY

mu.Host,

mu.User

* the list of privileges for a database:

SELECT

md.host `Host`,

md.user `User`,

md.db `Database`,

REPLACE(RTRIM(CONCAT(

IF(md.Select_priv = 'Y', 'Select ', ''),

IF(md.Insert_priv = 'Y', 'Insert ', ''),

IF(md.Update_priv = 'Y', 'Update ', ''),

IF(md.Delete_priv = 'Y', 'Delete ', ''),

IF(md.Create_priv = 'Y', 'Create ', ''),

IF(md.Drop_priv = 'Y', 'Drop ', ''),

IF(md.Grant_priv = 'Y', 'Grant ', ''),

IF(md.References_priv = 'Y', 'References ', ''),

IF(md.Index_priv = 'Y', 'Index ', ''),

IF(md.Alter_priv = 'Y', 'Alter ', ''),

IF(md.Create_tmp_table_priv = 'Y', 'Create_tmp_table ', ''),

IF(md.Lock_tables_priv = 'Y', 'Lock_tables ', ''),

IF(md.Create_view_priv = 'Y', 'Create_view ', ''),

IF(md.Show_view_priv = 'Y', 'Show_view ', ''),

IF(md.Create_routine_priv = 'Y', 'Create_routine ', ''),

IF(md.Alter_routine_priv = 'Y', 'Alter_routine ', ''),

IF(md.Execute_priv = 'Y', 'Execute ', ''),

IF(md.Event_priv = 'Y', 'Event ', ''),

IF(md.Trigger_priv = 'Y', 'Trigger ', '')

)), ' ', ', ') AS `Privileges`

FROM

mysql.db md

ORDER BY

md.Host,

md.User,

md.Db

* the list of privileges for tables:

SELECT

mt.host `Host`,

mt.user `User`,

CONCAT(mt.Db, '.', mt.Table_name) `Tables`,

REPLACE(mt.Table_priv, ',', ', ') AS `Privileges`

FROM

mysql.tables_priv mt

WHERE

mt.Table_name IN

(SELECT

DISTINCT

t.table_name `tables`

FROM

information_schema.tables AS t

WHERE

t.table_type IN

('BASE TABLE', 'SYSTEM VIEW', 'TEMPORARY', '') OR

t.table_type <> 'VIEW' AND

t.create_options IS NOT NULL

)

ORDER BY

mt.Host,

mt.User,

mt.Db,

mt.Table_name;

* the list of privileges for views:

SELECT

mv.host `Host`,

mv.user `User`,

CONCAT(mv.Db, '.', mv.Table_name) `Views`,

REPLACE(mv.Table_priv, ',', ', ') AS `Privileges`

FROM

mysql.tables_priv mv

WHERE

mv.Table_name IN

(SELECT

DISTINCT

v.table_name `views`

FROM

information_schema.views AS v

)

ORDER BY

mv.Host,

mv.User,

mv.Db,

mv.Table_name;

* the list of privileges for table columns:

SELECT

mtc.host `Host`,

mtc.user `User`,

CONCAT(mtc.Db, '.', mtc.Table_name, '.', mtc.Column_name) `Tables Columns`,

REPLACE(mtc.Column_priv, ',', ', ') AS `Privileges`

FROM

mysql.columns_priv mtc

WHERE

mtc.Table_name IN

(SELECT

DISTINCT

t.table_name `tables`

FROM

information_schema.tables AS t

WHERE

t.table_type IN

('BASE TABLE', 'SYSTEM VIEW', 'TEMPORARY', '') OR

t.table_type <> 'VIEW' AND

t.create_options IS NOT NULL

)

ORDER BY

mtc.Host,

mtc.User,

mtc.Db,

mtc.Table_name,

mtc.Column_name;

* the list of privileges for view columns:

SELECT

mvc.host `Host`,

mvc.user `User`,

CONCAT(mvc.Db, '.', mvc.Table_name, '.', mvc.Column_name) `Views Columns`,

REPLACE(mvc.Column_priv, ',', ', ') AS `Privileges`

FROM

mysql.columns_priv mvc

WHERE

mvc.Table_name IN

(SELECT

DISTINCT

v.table_name `views`

FROM

information_schema.views AS v

)

ORDER BY

mvc.Host,

mvc.User,

mvc.Db,

mvc.Table_name,

mvc.Column_name;

* the list of privileges for procedures:

SELECT

mp.host `Host`,

mp.user `User`,

CONCAT(mp.Db, '.', mp.Routine_name) `Procedures`,

REPLACE(mp.Proc_priv, ',', ', ') AS `Privileges`

FROM

mysql.procs_priv mp

WHERE

mp.Routine_type = 'PROCEDURE'

ORDER BY

mp.Host,

mp.User,

mp.Db,

mp.Routine_name;

* the list of privileges for functions:

SELECT

mf.host `Host`,

mf.user `User`,

CONCAT(mf.Db, '.', mf.Routine_name) `Procedures`,

REPLACE(mf.Proc_priv, ',', ', ') AS `Privileges`

FROM

mysql.procs_priv mf

WHERE

mf.Routine_type = 'FUNCTION'

ORDER BY

mf.Host,

mf.User,

mf.Db,

mf.Routine_name;

Netwerk & Server

How to check database

Check a Specific Table in a Database

If your application gives an error message saying that a specific table is corrupted, execute the mysqlcheck command to check that one table.

The following example checks employee table in thegeekstuff database.

# mysqlcheck -c thegeekstuff employee -u root -p
Enter password:
thegeekstuff.employee OK

Check All Tables in a Database

To check all the tables in a particular database, don’t specify the table name. Just specify the database name.

The following example checks all the tables in the alfresco database.

# mysqlcheck -c alfresco  -u root -p
Enter password:
alfresco.JBPM_ACTION                               OK
alfresco.JBPM_BYTEARRAY                            OK
alfresco.JBPM_BYTEBLOCK                            OK
alfresco.JBPM_COMMENT                              OK
alfresco.JBPM_DECISIONCONDITIONS                   OK
alfresco.JBPM_DELEGATION                           OK
alfresco.JBPM_EVENT                                OK
..

Check All Tables and All Databases

To check all the tables and all the databases use the “–all-databases” along with -c option as shown below.

# mysqlcheck -c  -u root -p --all-databases
Enter password:
thegeekstuff.employee                              OK
alfresco.JBPM_ACTION                               OK
alfresco.JBPM_BYTEARRAY                            OK
alfresco.JBPM_BYTEBLOCK                            OK
..
..
mysql.help_category
error    : Table upgrade required. Please do "REPAIR TABLE `help_category`" or dump/reload to fix it!
mysql.help_keyword
error    : Table upgrade required. Please do "REPAIR TABLE `help_keyword`" or dump/reload to fix it!
..

If you want to check all tables of few databases, specify the database names using “–databases”.

The following example checks all the tables in thegeekstuff and alfresco database.

# mysqlcheck -c  -u root -p --databases thegeekstuff alfresco
Enter password:
thegeekstuff.employee                              OK
alfresco.JBPM_ACTION                               OK
alfresco.JBPM_BYTEARRAY                            OK
alfresco.JBPM_BYTEBLOCK                            OK
..

Analyze Tables using Mysqlcheck

The following analyzes employee table that is located in thegeekstuff database.

# mysqlcheck -a thegeekstuff employee -u root -p
Enter password:
thegeekstuff.employee Table is already up to date

Internally mysqlcheck command uses “ANALYZE TABLE” command. While mysqlcheck is executing the analyze command the table is locked and available for other process only in the read mode.

Optimize Tables using Mysqlcheck

The following optimizes employee table that is located in thegeekstuff database.

# mysqlcheck -o thegeekstuff employee -u root -p
Enter password:
thegeekstuff.employee OK

Internally mysqlcheck command uses “OPTIMIZE TABLE” command. When you delete lot of rows from a table, optimizing it helps to get the unused space and defragment the data file. This might improve performance on huge tables that has gone through several updates.

Repair Tables using Mysqlcheck

The following repairs employee table that is located in thegeekstuff database.

# mysqlcheck -r thegeekstuff employee -u root -p
Enter password:
thegeekstuff.employee OK

Internally mysqlcheck command uses “REPAIR TABLE” command. This will repair and fix a corrupted MyISAM and archive tables.

Combine Check, Optimize, and Repair Tables

Instead of checking and repairing separately. You can combine check, optimize and repair functionality together using “–auto-repair” as shown below.

The following checks, optimizes and repairs all the corrupted table in thegeekstuff database.

# mysqlcheck -u root -p --auto-repair -c -o thegeekstuff

You an also check, optimize and repair all the tables across all your databases using the following command.

# mysqlcheck -u root -p --auto-repair -c -o --all-databases

If you want to know what the command is doing while it is checking, add the –debug-info as shown below. This is helpful while you are checking a huge table.

# mysqlcheck --debug-info -u root -p --auto-repair -c -o thegeekstuff employee
Enter password:
thegeekstuff.employee  Table is already up to date

User time 0.00, System time 0.00
Maximum resident set size 0, Integral resident set size 0
Non-physical pagefaults 344, Physical pagefaults 0, Swaps 0
Blocks in 0 out 0, Messages in 0 out 0, Signals 0
Voluntary context switches 12, Involuntary context switches 9

Additional Useful Mysqlcheck Options

The following are some of the key options that you can use along with mysqlcheck.

-A, –all-databases Consider all the databases
-a, –analyze Analyze tables
-1, –all-in-1 Use one query per database with tables listed in a comma separated way
–auto-repair Repair the table automatically it if is corrupted
-c, –check Check table errors
-C, –check-only-changed Check tables that are changed since last check
-g, –check-upgrade Check for version dependent changes in the tables
-B, –databases Check more than one databases
-F, –fast Check tables that are not closed properly
–fix-db-names Fix DB names
–fix-table-names Fix table names
-f, –force Continue even when there is an error
-e, –extended Perform extended check on a table. This will take a long time to execute.
-m, –medium-check Faster than extended check option, but does most checks
-o, –optimize Optimize tables
-q, –quick Faster than medium check option
-r, –repair Fix the table corruption

Netwerk & Server

Nagios

check_mk

download new version: https://checkmk.com/download.php?HTML=yes
# yum/zypper install <##>
# omd stop
# omd update
# omd start

Introduction

This guide is intended to provide you with simple instructions on how to install Nagios from source (code) on openSUSE and have it monitoring your local machine inside of 20 minutes. No advanced installation options are discussed here - just the basics that will work for 95% of users who want to get started.

These instructions were written based on an openSUSE 10.2 installation.

Required Packages

Make sure you've installed the following packages on your openSUSE installation before continuing. You can use yast to install packages under openSUSE.

apache2
C/C++ development libraries

Create Account Information

Become the root user.

su -l

Create a new nagios user account and give it a password.

/usr/sbin/useradd -m nagios
passwd nagios

Create a new nagios group. Add the nagios user to the group.

/usr/sbin/groupadd nagios
/usr/sbin/usermod -G nagios nagios

Create a new nagcmd group for allowing external commands to be submitted through the web interface. Add both the nagios user and the apache user to the group.

/usr/sbin/groupadd nagcmd
/usr/sbin/usermod -a -G nagcmd nagios
/usr/sbin/usermod -a -G nagcmd wwwrun

Download Nagios and the Plugins

Create a directory for storing the downloads.

mkdir ~/downloads
cd ~/downloads

Download the source code tarballs of both Nagios and the Nagios plugins (visit http://www.nagios.org/download/ for links to the latest versions). These directions were tested with Nagios 3.1.1 and Nagios Plugins 1.4.11.

wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.3.tar.gz
wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.11.tar.gz

Compile and Install Nagios

Extract the Nagios source code tarball.

cd ~/downloads
tar xzf nagios-3.2.3.tar.gz
cd nagios-3.2.3

Run the Nagios configure script, passing the name of the group you created earlier like so:

./configure --with-command-group=nagcmd

Compile the Nagios source code.

make all

Install binaries, init script, sample config files and set permissions on the external command directory.

make install
make install-init
make install-config
make install-commandmode

Don't start Nagios yet - there's still more that needs to be done...

Customize Configuration

Sample configuration files have now been installed in the /usr/local/nagios/etc directory. These sample files should work fine for getting started with Nagios. You'll need to make just one change before you proceed...

Edit the /usr/local/nagios/etc/objects/contacts.cfg config file with your favorite editor and change the email address associated with the nagiosadmin contact definition to the address you'd like to use for receiving alerts.

vi /usr/local/nagios/etc/objects/contacts.cfg

Configure the Web Interface

Install the Nagios web config file in the Apache conf.d directory.

make install-webconf

Create a nagiosadmin account for logging into the Nagios web interface. Remember the password you assign to this account - you'll need it later.

htpasswd2 -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Restart Apache to make the new settings take effect.

service apache2 restart
or
systemctl restart apache2

Note Note: Consider implementing the ehanced CGI security measures described below to ensure that your web authentication credentials are not compromised.

Compile and Install the Nagios Plugins

Extract the Nagios plugins source code tarball.

cd ~/downloads
tar xzf nagios-plugins-1.4.11.tar.gz
cd nagios-plugins-1.4.11

Compile and install the plugins.

./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install

Start Nagios

Add Nagios to the list of system services and have it automatically start when the system boots.

chkconfig --add nagios
chkconfig nagios on

Verify the sample Nagios configuration files.

/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

If there are no errors, start Nagios.

service nagios start
or
systemctl start nagios

You should now be able to access the Nagios web interface at the URL below. You'll be prompted for the username (nagiosadmin) and password you specified earlier.

http://localhost/nagios/

Click on the "Service Detail" navbar link to see details of what's being monitored on your local machine. It will take a few minutes for Nagios to check all the services associated with your machine, as the checks are spread out over time.

Other Modifications

Make sure your machine's firewall rules are configured to allow access to the web server if you want to access the Nagios interface remotely.

You can do this by:

Opening the control center
Select 'Open Administrator Settings' to open the YaST administrator control center
Select 'Firewall' from the 'Security and Users' category
Click the 'Allowed Services' option in the Firewall Configuration window
Add 'HTTP Server' to the allowed services list for the 'External Zone'
Click 'Next' and 'Accept' to activate the new firewall settings

Configuring email notifications is outside the scope of this documentation. Refer to your system documentation, search the web, or look to the Nagios Support Portal or Nagios Community Wiki for specific instructions on configuring your openSUSE system to send email messages to external addresses.

Security

This is intended to be an introduction for implementation of stronger authentication and server security focused around the CGI web interface.

Stronger Authentication using Digest Authentication.

If you have followed the quickstart guides, chances are that you are using Apache's Basic Authentication. Basic Authentication will send your username and password in "clear text" with every http request. Consider using a more secure method of authentication such as Digest Authentication which creates a MD5 Hash of your username and password to send with each request.

Forcing TLS/SSL for all Web Communication.

Apache provides TLS/SSL through the mod_ssl module. TLS/SSL provides a secure tunnel between the client and server that prevents eavesdropping and tampering using strong publickey/privatekey cryptography.

Locking Down Apache Using Access Controls.

Consider locking down access to the Nagios box to your IP address, IP address range, or IP subnet. If you require access outside your network you could use VPN or SSH Tunnels. This is a easy and strong to limit access to HTTP/HTTPS on your system.

Implementing Digest Authentication

The implementation of Digest Authentication is simple. You will have to create the new type of password file using the 'htdigest' tool, then modify the Apache configuration for nagios (typically /etc/httpd/conf.d/nagios.conf).

Create a new passwords file using the 'htdigest' tool. The difference that you will notice if you are familiar with 'htpasswd' tools is the requirement to supply a 'realm' argument. Where 'realm' in this case refers to the value of the 'AuthName' directive in the Apache configuration.

htdigest -c /usr/local/nagios/etc/.digest_pw "Nagios Access" nagiosadmin

Next, edit the Apache configuration file for Nagios (typically /etc/httpd/conf.d/nagios.conf) using the following example.

## BEGIN APACHE CONFIG SNIPPET - NAGIOS.CONF

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">

   Options ExecCGI

   AllowOverride None

   Order allow,deny

   Allow from all

   AuthType Digest

   AuthName "Nagios Access"

   AuthDigestFile /usr/local/nagios/etc/.digest_pw

   Require valid-user

</Directory>



Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">

   Options None

   AllowOverride None

   Order allow,deny

   Allow from all

   AuthType Digest

   AuthName "Nagios Access"

   AuthDigestFile /usr/local/nagios/etc/.digest_pw

   Require valid-user

</Directory>

## END APACHE CONFIG SNIPPETS

Then, restart the Apache service so the new settings can take effect.

/etc/init.d/httpd restart

Implementing Forced TLS/SSL

Make sure you've installed Apache and OpenSSL. By default you should have mod_ssl support if you are still having trouble you may find help reading Apache's TLS/SSL Encryption Documentation.

Next, verify that TLS/SSL support is working by visiting your Nagios Web Interface using HTTPS (https://your.domain/nagios). If it is working you can continue on to the next steps that will force using HTTPS and block all HTTP requests for the Nagios Web Interface. If you are having trouble visit Apache's TLS/SSL Encryption Documentation and Google for troubleshooting your specific Apache installation. Next, edit the Apache configuration file for Nagios (typically /etc/httpd/conf.d/nagios.conf) by adding the 'SSLRequireSSL' directive to both the 'sbin' and 'share' directories.

## BEGIN APACHE CONFIG SNIPPET - NAGIOS.CONF

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">

   ...

   SSLRequireSSL

   ...

</Directory>



Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">

   ...

   SSLRequireSSL

   ...

</Directory>

## END APACHE CONFIG SNIPPETS

Restart the Apache service so the new settings can take effect.

/etc/init.d/httpd restart

Implementing IP subnet lockdown

The following example will show how to lock down Nagios CGIs to a specific IP address, IP address range, or IP subnet using Apache's access controls.

Edit the Apache configuration file for Nagios (typically /etc/httpd/conf.d/nagios.conf) by using the 'Allow', 'Deny', and 'Order' directives using the following as an example.

## BEGIN APACHE CONFIG SNIPPET - NAGIOS.CONF

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">

   ...

   AllowOverride None

   Order deny,allow

   Deny from all

   Allow from 127.0.0.1 10.0.0.25       # Allow single IP addresses

   Allow from 10.0.0.0/255.255.255.0        # Allow network/netmask pair

   Allow from 10.0.0.0/24           # Allow network/nnn CIDR spec

   ...

</Directory>



Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">

   ...

   AllowOverride None

   Order deny,allow

   Deny from all

   Allow from 127.0.0.1 10.0.0.25       # Allow single IP addresses

   Allow from 10.0.0.0/255.255.255.0        # Allow network/netmask pair

   Allow from 10.0.0.0/24           # Allow network/nnn CIDR spec

   ...

</Directory>

## END APACHE CONFIG SNIPPET

Important Notes

Digest Authentication sends data in the clear but not your username and password.
Digest Authentication is not as universally supported as Basic Authentication.
TLS/SSL has potential for "man-in-the-middle attacks". MITM attacks are vulnerable if an attacker is able to insert itself between the server and client such as in a Phishing attack, ISP monitoring, or corporate LAN firewall certificate resigning. So read up on certificate verification!
Apache access controls only protect the HTTP/HTTPS protocols. Look into IPtables for strong system wide firewall control.
Most importantly, Security is a moving target so stay informed and do research! Perhaps by listening to a Podcast such as "Security Now!".

Netwerk & Server

Nmap

brute force test wordpress

Netwerk & Server

Brute force test wordpress

nmap -p 80 --script http-wordpress-brute --script-args http-wordpress-brute.threads=4,http-wordpress-brute.uri="/blog/wp-login.php"

Netwerk & Server

Bluetooth

To enable the (powered down) bluetooth adapter on openSUSE 13.2:

rfkill unblock bluetooth

blueproximity

tweaking your settings should do the trick. Actually that's why I implemented a kind of testing mode. Open the preferences. Move to the second tab. The last control on that page "Measured atm (at the moment) shows the measured state. You can click on "Reset min/max" and leave the window open. If the window is open BlueProximity enters the simulation state which will not execute the action commands. It still measures the distances (on contrary to the pause mode which will completely disconnect the phone) and shows them at that control. Please note the text below the bar, which states "min" and "max" values - those are the min/max values of the measured distances after you pressed the Reset button last time.

This information should give you a hint which values are to expect while you normally work in front of your computer. Remember to use all "poses" you normally use while using your computer. You should stay in those poses for at least 10 seconds so that the rssi value regulates on the new situation. That should give you realistic values for the locking distance. Use a value of at least 2 above the measured value and see if that already gives a secure setting for the distance.

Also set the unlock distance to a value at least 2 smaller than the lock distance. Normally half of the lock distance gives a good result. You should play with the timer settings too. Don't put your lock timer to a value less than 5, at this moment my values are lock dist=7,dur=10. That is because I found out that sometimes unexpected com problems seem to exist between my phone and my pc - the rssi value drops to an unreasonable 8 but only for 5 seconds, it comes back to normal then which will never result in a value above 6 for longer than 10 seconds. My unlock time is also not set to 0 but to 2, the distance to 4. That way if I come back to my pc, the first of the two seconds below 5 is usually already passed and it takes about a second for me to sit down and once I sit down it unlocks almost immediately.

This should also fix the screensaver problem as there is no immediate unlock possible. Please do note that using a fade out time for you screensaver, it should be set at least one second below the unlock timer so that the unlocking is never done while the locking is still in progress. It once rendered my whole desktop faded to grey which was eyefriendly for that time of the day but it wouldn't help in the office :-)

The battery drain should be unnotable to phones that already had bluetooth activated. The explicit energy drain for an open bluetooth connection is specified to be 1 mW, my nokia has a 700mAh battery so it is ridiculously low in comparison :-) In other words, if you have bluetooth activated already you should not notice any difference at all. And btw, the visibility setting in your phone also has no effect on the battery too. Even though you would need to activate it only once to find your phone. After the mac address is entered you can switch your phone back to invisible if you like. This raises your phone's security a little bit. Hopefully this helps,

Netwerk & Server

KVM

Start a vm as non-root:

# virsh -c qemu:///system start tumbleweed

and view it:

# virt-viewer -c qemu:///system tumbleweed

Netwerk & Server

Raspberry Pi

enable 3d acceleration

Netwerk & Server

X11 forwarding for sudo users

Can’t start X11 applications after “su” or “su -” to another user

To get access to the X client applications such as system-config-date, xclock, vncviewer we need to export the DISPLAY settings of a remote host to the local server. This is commonly done using below commands.

# ssh root@remotehost

remotehost# export DISPLAY=x.x.x.x:y.y

Where x.x.x.x:y.y – is the display settings of the system from which you connected to the remote host.

You can also use the -X option with ssh to directly export the DISPLAY on the remote host.

# ssh -X root@remotehost

But now if you try to switch to another user on the remote system and export the display again, you would get and error – “Error: Can’t open display:”.

# ssh -X root@remotehost
# su - [username]
# export DISPLAY=x.x.x.x:y.y
# xclock
Error: Can't open display: x.x.x.x:y.y

X11 forwarding for sudo users

Just setting the DISPLAY is not enough. X authentication is based on cookies, so it’s necessary to set the cookie used by the user that initiated the connection. The following procedure allows a sudo user to use the ssh based X11 tunnel:

1. Connect the remote host using the -X option with ssh.

# ssh -X root@remote-host

2. Now list the coockie set for the current user.

# xauth list $DISPLAY
node01.thegeekdiary.com/unix:10 MIT-MAGIC-COOKIE-1 dacbc5765ec54a1d7115a172147866aa
# echo $DSIPLAY
localhost:10.0

3. Switch to another user account using sudo. Add the cookie from the command output above to the sudo user.

# sudo su - [user]
# xauth add node01.thegeekdiary.com/unix:10 MIT-MAGIC-COOKIE-1 dacbc5765ec54a1d7115a172147866aa

4. Export the display from step 2 again for the sudo user. Try the command xclock to verify if the x client applications are working as expected.

# export DISPLAY=localhost:10.0
# xclock

Netwerk & Server

Ssh keygen

# ssh-keygen -t ecdsa -b 521

Netwerk & Server

Compare Apache conf files and Nginx conf files

APACHE:

Apache Nginx test

<VirtualHost *:80> server {
listen 80;

ServerName yoursite.com
ServerAlias www.yoursite.com server_name yoursite.com www.yoursite.com;

DocumentRoot /path/to/root root /path/to/root;

AllowOverride All <span class=“highlight”>(No Available Alternative)</span>

DirectoryIndex index.php index index.php;

ErrorLog /path/to/log error_log /path/to/log error;

CustomLog /path/to/log combined access_log /path/to/log main;

Alias /url/ “/path/to/files” location /url/ {
<Directory “/path/to/files”> alias /path/to/files;

NGINX:

server {
listen 8000; # We’re deliberately leaving this as-is to avoid conflict at the moment

root /var/www;
server_name <span class="highlight">your_site.com</span> www.<span class="highlight">your_site.com</span>;

location / {
    try_files $uri $uri/ /index.html;
}

location ~ \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_index index.php;
    include fastcgi_params;
}

location /doc/ {
    alias /usr/share/doc/;
    autoindex on;
    allow 127.0.0.1;
    deny all;
}

location ~/\.ht {
    deny all;
}
}

Netwerk & Server

Enable wifi after suspend-resume

To enable wifi after resume, when it is turned off

Create a systemd service “root-resume.service” in /etc/systemd/system with content:

[Unit]
Description=Local system resume actions
After=suspend.target

[Service]
Type=simple
ExecStart=/usr/bin/nmcli radio wifi on

[Install]
WantedBy=suspend.target

and enable the newly created service:

# systemctl enable root-resume.service

Netwerk & Server

POSTFIX

Inleiding

Met het programma Postfix op een linux systeem kan mail verzonden en verstuurd worden. Er zijn een paar instellingen die soms gedaan moeten worden. Tevens het gebruik van het lokale mailprogramma wordt in deze hoofdstuk uitgelegd (mail,nail,mailx)

De instellingen

De instellingen die belangrijk zijn om via Postfix mail te kunnen versturen staan in '/etc/postfix/main.cf'. De volgende instellingen zijn belangrijk:

myhostname="fully qualified hostname"
relayhost=bla.nl

De laatste instelling is verplicht. Indien deze parameter niet gevuld is, wordt de mail niet verstuurd naar de ontvanger (postfix zoekt dan zelf de relayhost op via het 'mx-record' van het domein, wat een mail relay host teruggeeft die niet relayed -XXXXX )

Mail manipulaties

Nu kan er dus mail verstuurd worden vanaf de host met het commando 'mail' of 'nail'. Dit kan interactief of scriptsgewijs. Een aantal voorbeeldscriptjes zijn:

nail -s "subject" -r "afzender" "mailadres" -a "attachment" << EOF bla body bla groetje EOF
mail bla@amsterdam.nl -s 'hallootjes' -r joop@bla.tk < /proc/cpuinfo
mail bla@amsterdam.nl -s 'hallootjes' -r joop@bla.tk -a attachment < /proc/cpuinfo

of simpeler:

mail -s "subject"  "mailadres"  bewaar de gelezen mail, de uitvoer geeft aan waar de mail terechtkomt (bijv. '/root/mbox')
help    --> geeft aan hoe je een mail kan versturen
list    --> laat alle commando's zien (een soort help maar dan uitgebreider)
next    --> ga naar het volgende mailtje
d 1     --> delete mail 1
d 1-300 --> delete mail 1 t/m 300
header  --> laat de lijst met mail zien
q       --> verlaat de applicatie

De mailboxen

De postfix mailbox is opgeslagen in de directory '/var/spool/mail/"username". De gelezen en opgeslagen mail (gaat automatisch bij het afsluiten) staat in "user_home_dir"/mbox.

De postfix deliveryqueue legen/flushen

#postqueue -f <-- waarschijnlijk niet het goede commando
#postsuper -d

Alles verwijderen:

#postsuper -d ALL

Het queueID kun je vinden met

#postqueue -p

Alle systeem mail for root verwijderen

echo 'd*' | mail -N

De postfix queue bekijken

#mailq

#postqueue -p

Een werkende postfix configuratiefile main.cf

Hier volgt een goed werkende file voor de transparent mail relay host cups:

relay_domains = $mydestination, hash:/etc/postfix/relay
virtual_alias_maps = hash:/etc/postfix/virtual
#virtual_uid_maps = static:303
#virtual_gid_maps = static:303
#virtual_minimum_uid = 303
#virtual_mailbox_base = /srv/maildirs
#virtual_mailbox_domains = mysql:/etc/postfix/mysql_virtual_domains_maps.cf
#virtual_mailbox_limit = 0
#virtual_mailbox_limit_inbox = no
#virtual_mailbox_limit_maps = hash:/etc/postfix/vquota
#virtual_mailbox_limit_override = yes
#virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
#virtual_transport = virtual
## Additional for quota support
#virtual_create_maildirsize = yes
#virtual_mailbox_extended = yes
### Needs Maildir++ compatible IMAP servers, like Courier-IMAP
#virtual_maildir_filter = yes
#virtual_maildir_filter_maps = hash:/etc/postfix/vfilter
#virtual_mailbox_limit_maps = mysql:/etc/postfix/mysql_virtual_mailbox_limit_maps.cf
#virtual_mailbox_limit_override = yes
#virtual_maildir_limit_message = Sorry, the user's maildir has overdrawn his diskspace quota, please try again later.
#virtual_maildir_limit_message_maps = hash:/etc/postfix/vmsg
#virtual_overquota_bounce = yes
#virtual_trash_count = yes
#virtual_trash_name = ".Trash"
############################################################
# End MySQL from postfixwiki.org
############################################################

inet_protocols = all
biff = no
mail_spool_directory = /var/mail
canonical_maps = hash:/etc/postfix/canonical
virtual_alias_domains = hash:/etc/postfix/virtual
relocated_maps = hash:/etc/postfix/relocated
transport_maps = hash:/etc/postfix/transport
sender_canonical_maps = hash:/etc/postfix/sender_canonical
masquerade_exceptions = root
masquerade_classes = envelope_sender, header_sender, header_recipient
myhostname = bla.wzs.amsterdam.nl
delay_warning_time = 1h
message_strip_characters = \0
program_directory = /usr/lib/postfix
inet_interfaces = xxx
#inet_interfaces = localhost
masquerade_domains = 
mydestination = $myhostname, localhost.$mydomain
defer_transports = 
mynetworks = xxx
#mynetworks_style = subnet
disable_dns_lookups = no
relayhost = smtprelay.basis.lan
content_filter = 
mailbox_command = 
mailbox_transport = 
strict_8bitmime = no
disable_mime_output_conversion = no
smtpd_sender_restrictions = hash:/etc/postfix/access
smtpd_client_restrictions = permit_mynetworks 
smtpd_helo_required = no
smtpd_helo_restrictions = 
strict_rfc821_envelopes = no
smtpd_recipient_restrictions = permit_mynetworks,reject_unauth_destination
smtp_sasl_auth_enable = no
smtpd_sasl_auth_enable = no
smtpd_use_tls = no
smtp_use_tls = no
smtp_enforce_tls = no
alias_maps = hash:/etc/aliases
mailbox_size_limit = 0
message_size_limit = 0

Netwerk & Server

How to make a webcam page with Apache on Linux (obsolete)

How to make a page on your webserver displaying a picture of your webcam using Linux and Apache

This document describes, in short, how to make a page on your webserver to display a semi-continuous image of your webcam. This document is in fact, the documentation of such an installation on my machine running openSUSE 11.2 x86_64

The webcam involved is a Philips ToUCam Pro. In this example, it is connected directly to the webserver. But it is possible to have the image uploaded from another computer as well.

This solution uses a 2 stage approach, which makes it modular. The first stage/module captures images from the device using the commandline tool webcam and places the image in a directory, and the second stage/module is the HTML-page that publishes this image on a regular interval using a java applet (or you can go for a more simple solution and use a static page).

1st step: Check webcam.

check dmesg

pwc: Philips PCVC740K (ToUCam Pro)/PCVC840 (ToUCam II) USB webcam detected.

2nd step: install motion.

buildservice /etc/motion/motion.conf section image file output

output normal on -> off

section live webcam server

webcam port 8081
webcam quality 80
webcam localhost off

/var/run/motion/motion.pid

#mkdir /var/run/motion
#chmod 777 /var/run/motion

Netwerk & Server

Enable monitoring of MD (software RAID) devices

After you have created the RAID (via YAST):

#insserv -d mdadmd

to have mdadm run as a daemon in the background in the deafult runlevels (usually 3 and 5) monitoring your devices. Or you can run it manually using

#mdadm --monitor --scan --daemonise > /var/run/mdadm

this also writes the pid to /var/run/mdadm file.

Also, check it by running:

#chkconfig -l mdadmd
mdadmd 0:off 1:off 2:off 3:on 4:off 5:on 6:off

Netwerk & Server

Database dump & restore

make database dump:

mysqldump -u root -p –all-databases > pvdm.sql
mysqldump -u root -p wikidb > pvdm.sql

restore the dump:

#mysql -u root -p
mysql> create database xyz;
mysql> quit;
#mysql -u root -p [dbname] < pvdm.sql

delete database:

#mysql -u root -p
mysql> drop database xyz;

Netwerk & Server

Stop SSH brute force attacks

Stop ssh brute force attack using SuSEfirewall

Edit /etc/sysconfig/SuSEfirewall2:

do not open ssh ports here

FW_SERVICES_EXT_TCP=""
FW_CONFIGURATIONS_EXT=""

add this rule

FW_SERVICES_ACCEPT_EXT="0.0.0.0/0,tcp,22,,hitcount=3,blockseconds=60,recentname=ssh"

Restart firewall:

rcSuSEfirewall2 restart

Now attacker will just have three attempts to break in.

Netwerk & Server

Securing the initial mysql installation

When mysql is installed, the grant tables are created.

accounts with the name root are created. The initial password are empty.
- On Unix, both root accounts are for connections from the local host. Connections must be made from the local host by specifying a host name of localhost for one of the accounts, or the actual host name or IP number for the other

two anonymous-user accounts are created, each with an empty user name. The anonymous accounts have no password, so anyone can use them to connect to the MySQL server.
- On Unix, both anonymous accounts are for connections from the local host. Connections must be made from the local host by specifying a host name of localhost for one of the accounts, or the actual host name or IP number for the other. These accounts have all privileges for the test database and for other databases with names that start with test_.

root account password assignment:

shell> mysql -u root -p
mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd');
mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd');

: In the second SET PASSWORD statement, replace host_name with the name of the server host. This is the same host name that you used when you assigned the anonymous account passwords.

: If the user table contains an account with User and Host values of 'root' and '127.0.0.1', use an additional SET PASSWORD statement to set that account's password:

mysql> SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd');

Anonymous Account Removal

: If you prefer to remove the anonymous accounts instead, do so as follows:

shell> mysql -u root
mysql> DROP USER ''; (zonder spatie)

: The DROP statement applies both to Windows and to Unix.

new users

After connecting to the server as root, you can add new accounts. The following statements use GRANT to set up four new accounts:

mysql> CREATE USER 'monty'@'localhost' IDENTIFIED BY 'some_pass';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'localhost'
-> WITH GRANT OPTION;
mysql> CREATE USER 'monty'@'%' IDENTIFIED BY 'some_pass';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%'
-> WITH GRANT OPTION;
mysql> CREATE USER 'admin'@'localhost';
mysql> GRANT RELOAD,PROCESS ON *.* TO 'admin'@'localhost';
mysql> CREATE USER 'dummy'@'localhost';

Netwerk & Server

Install a PXE boot server on 10.3

REDIRECT Install a PXE boot server on 11.1

Netwerk & Server

How to make a webcam page with Apache on Linux

REDIRECT How to make a webcam page with Apache on Linux (obsolete)

Netwerk & Server

How to automatically log in to remote ssh servers using public/private keypair

As a user:

~>ssh-keygen -t dsa

and enter your way through the prompts

The keys are stored in the .ssh/ directory in your home directory.

-rw------- 1 pm users 668 2009-10-24 15:58 id_dsa
-rw-r--r-- 1 pm users 601 2009-10-24 15:58 id_dsa.pub

id_dsa.pub is the public keyfile we are gonna place on the remote server.

ssh-copy-id -i ~/.ssh/id_dsa.pub remoteserver

Enter your password for the remote server.

You should get:

Now try logging into the machine, with "ssh 'remoteserver'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.

Now try it for yourself:

ssh remoteserver

Et voila!

Netwerk & Server

How to make an encrypted directory

If you want to make an encrypted directory, for instance when you want to store private information, and you want to access it seamless on your computer, this is how to do it. It will be available after mounting it with a password.

This wiki is not about an encrypted filesystem.

Installation

Install the package 'encfs'.

OpenSUSE:

#zypper in encfs

Configuration

Now you have to create 2 directories.

One will contain the encrypted, unreadable data. The files will be stored here. But do nothing with this directory.
The second will be used as a sort of 'mount point', where the unencrypted data will appear. You must read and write this location.

You can use any name. Here I have chosen the following:

#mkdir ~/.encrypted ~/unencrypted

I choose a name starting with a period ('.') for the encrypted dir because this will hide it from normal viewing, as there is nothing to see anyway :)

Now start the installation:

#encfs ~/.encrypted ~/unencrypted

Creating new encrypted volume.
Please choose from one of the following options:
 enter "x" for expert configuration mode,
 enter "p" for pre-configured paranoia mode,
 anything else, or an empty line will select standard mode.

Press [enter] for standard mode

Standard configuration selected.

Configuration finished.  The filesystem to be created has
the following properties:
Filesystem cipher: "ssl/aes", version 2:2:1
Filename encoding: "nameio/block", version 3:0:1
Key Size: 192 bits
Block Size: 1024 bytes
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.
File holes passed through to ciphertext.

Now you will need to enter a password for your filesystem.
You will need to remember this password, as there is absolutely
no recovery mechanism.  However, the password can be changed
later using encfsctl.

New Encfs Password:
Verify Encfs Password:

And there you have it!

Usage

To use the encrypted directory, place some files in the unencrypted directory. You notice that the same amount of files are created in the encrypted directory, but with encrypted names and encrypted contents.

You can use the unencrypted dir as you would with any normal dir.

If you want to stop using the files, you can logout or unmount manually.

Make sure that it is not the current directory.

#fusermount -u ~/unencrypted

To use it again:

#encfs ~/.encrypted ~/unencrypted

And that's it!

Netwerk & Server

How to repair a crashed table

I had a crashed searchindex table in my MediaWiki, so I:

Connect to the server

#mysql -u root -p

Connect to the database

mysql> use wikidb;

check data

fix it

Netwerk & Server

How to test network speed

Install iperf http://software.opensuse.org/search http://software.opensuse.org/search
start on server:

# iperf -s

start on client:

# iperf -c [ip.of.server.address] -r -t 10

-r: test reverse also

-t: time

Netwerk & Server

How to convert an existing ext3 filesystem to ext4

Dismount the filesystem
tune2fs -O extents,uninit_bg,dir_index /dev/DEV
e2fsck -fpDC0 /dev/DEV

Netwerk & Server

Using bridged networking in VirtualBox

If you get an error when using bridged networking, type:

# insmod /lib/modules/2.6.31.8-0.1-desktop/updates/vboxnetflt.ko

That could help. Of course, substitute your current kernel.

Netwerk & Server

Installation of openfire jabber xmpp server on openSUSE server

download the rpm form the website
install the rpm with

# zypper in openfire.rpm

tidy the start script /etc/init.d/openfire. Paste this at the beginning of the file, replacing what is already there....

#! /bin/sh                                                                               
#                                                                                        
# openfire      Stops and starts the Openfire XMPP service.                              
#                                                                                        
### BEGIN INIT INFO                                                                      
# Provides:          openfire                                                            
# Required-Start:    $network                                                            
# Required-Stop:     $network                                                               
# Default-Start:     3 5                                                                    
# Default-Stop:                                                                             
# Description:                                                                              
# description:       Openfire is an XMPP server, which is a server that facilitates \       
#                    XML based communication, such as chat.                                 
# config:            /opt/openfire/conf/openfire.xml                                        
# config:            /etc/sysconfig/openfire                                                
# pidfile:           /var/run/openfire.pid                                                  
# Short-Description: XMPP server                                                            
### END INIT INFO

start the server

# /etc/init.d/openfire start
# /etc/init.d/openfire status

go to the admin url http://:9090

fill in the right data

make it start automagically

# chkconfig -a openfire

Troubleshooting

Openfire admin password

Find the sparkweb.xml file in the Conf directory, open it in your favorite text editor and place the following:

<admin>
    <authorizedUsernames>joe</authorizedUsernames>
</admin>

Then restart the service!!

Where "joe" is any user you want to use as the admin. This will allow you to login with an account you know.

If this doesn't work, open the same sparkweb.xml file, find the entry near the bottom that says true and change it to false.

Stop and start the service, open the admin console and you will be prompted to begin the setup again. You previous settings will be saved and you can press "save & continue" for most settings. When it asks for the admin account, fix it there.

Netwerk & Server

Apache etc. (joomla etc.)

REDIRECT Apache en HTML

Netwerk & Server

How to secure an initial MySQL installation

REDIRECT OLD:How to secure an initial MySQL installation

Netwerk & Server

Enlarge disks by removing reserve space on ext2/3/4

Five percent of a ext2/3/4 filesysteem is reserved for root.

With tune2fs this can be tweaked, even after the filesystem is created.

# tune2fs -m 1 /dev/sda1

this reserves 1 percent.

# tune2fs -m 0.2 /dev/sda1

this reserves 0.2 percent

On a filesystem that has no systemfiles, like homedirs, there is no use for any reserved space. You can make this 0 percent.

Netwerk & Server

Allowing access from other hosts

To allow access from your workstation (to use mysqladministrator?) you must allow access.

mysql> update user set host='10.125.0.3' where user='root' and host='localhost';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> flush privileges;
Query OK, 0 rows affected (0.05 sec)

Netwerk & Server

Install openSUSE via SSH

To install a server over SSH do the following:

insert CD/DVD into the machine and boot
on the kerneloptionsline type:

usessh=1 sshpassword=sdrtjgbskdhg hostip=1.1.1.1 netmask=255.255.255.0 gateway=1.1.1.1

boot the machine
you van continue the installation using an SSH session from a workstation.

Netwerk & Server

SSH filesystem

SSH Filesystem

Another way to get at files on remote hosts? The SSH Fuse filesystem. This effectively mounts remote filesystems using SSH. It's a bit complicated, but not bad. Here's how we do it (it's easier after the first time):

Make sure you have the sshfs and FUSE packages installed. If you install sshfs on Ubuntu, for instance, it should pull in the necessary packages.
Make sure that the fuse module is loaded. Check using lsmod | grep fuse and if it isn't loaded, run sudo modprobe fuse.
Make sure you're in the fuse group. May require a log out and in again. (adduser username fuse or just append your username to the fuse line in /etc/group.)
Now, make sure you have a directory to serve as a local mount point. I usually just use the server name. So I might make a local directory under my home directory called "kang" for a remote server called kang.zonker.net.
To mount the remote filesystem, run: sshfs hostname:/directory localdirectory. For example, I might run: sshfs jzb@kang.zonker.net:/home/jzb/ kang .

That's it. Now you can browse the remote filesystem as if it were local. Naturally, like copying files over SFTP or scp, it's going to be slower. But this makes remote file management dead easy.

Overig

Main Page

Categories:

Overig

Quad

REDIRECT Audio

Overig

Kmail

Transfer mail and settings to another computer (or another user account on the same machine)

Solution: The messages are typically in ~/.kde/share/apps/kmail/.

For very old installations of KMail, the messages can also be in ~/Mail.

Note that KMail uses hidden sub-directories inside that directory, so you need to make sure to copy hidden directories as well.

For settings you will need to copy the following files:

~/.kde/share/config/kmailrc,
~/.kde/share/config/mailtransports, (since KDE 4.0)
~/.kde/share/config/emaildefaults and
~/.kde/share/config/emailidentities .

Your address book is usually stored in ~/.kde/share/apps/kabc/.

Calendar data is in ~/.kde/share/apps/korganizer

Be aware that some distributions use ~/.kde4 instead of ~/.kde/ for their KDE configuration data.

Look at the output of the command kde4-config --localprefix if in doubt.

From version 4.4 you may have some Akonadi-controlled entries that also need to be kept. Add to the above list -

Everything under ~/.local/share/
Everything under ~/.config/akonadi/.
~/.kde/share/config/nepomukserverrc
Everything under ~/.kde/share/apps/nepomuk/ (KAddressBook stores contact groups in Nepomuk.)

If you plan to use an USB stick as transport medium for your data, be sure to format to a linux FileSystem.

Overig

How to start qbittorrent

rc.d/after.local:su - qbtuser -c "/usr/bin/qbittorrent-nox -d"

Overig

Create a bootable windows 10 USB stick from linux

Creating a bootable Windows 10 USB on linux

10 Aug 2016 - by 'Maurits van der Schee'

Whenever I have to help somebody with Windows 10 I find myself making a bootable Windows 10 USB drive. If you are on Windows this is provided via a menu option, but on Ubuntu Linux this requires some commands. This post will explain in detail how to do this.

GPT partition table with FAT32 partition

Modern systems support UEFI booting and this differs from traditional BIOS in that it does not read the boot sector. Instead it will look in the /efi/ directory in a FAT32 partition of a drive with GPT partitioning. There are no partition flags (like 'boot') necessary either.

Getting the official Windows 10 ISO image

You can download the official Windows 10 ISO image from Microsoft by visiting:

https://www.microsoft.com/en-us/software-download/windows10ISO

But you must be on Linux (or pretend to be using user agent switcher) to download the file.

Writing to the USB drive

Your USB drive has a drive letter in Linux typically something like 'b', which leads to a path to the device of /dev/sdb and a partition path for the first partition of /dev/sdb1. To avoid you screwing up your computer I use 'h', which you should replace in the commands below. You can use the 'gparted' graphical tool to find the drive letter of your USB drive.

If your USB drive is behaving weird, then you can reset it by issuing the following command:

# dd if=/dev/zero of=/dev/sdh bs=512 count=1

Normally this is not nessecary. Then to create a new layout on the drive:

# fdisk /dev/sdh

This is what you should enter in fdisk:

Press 'g' to create a new GPT, Press 'n' to create a new partition and accept the defaults, Press 't' to choose the partition type, Enter '11' for the 'Microsoft basic data' type. Press 'w' to write the changes to disk.

The following command will format the first partition with FAT32.

# mkdosfs -F32 /dev/sdh1

Now we need to mount the downloaded ISO file:

# mount Downloads/Win10_1607_Dutch_x64.iso /mnt

And copy all contents to the USB drive (this takes up to 15 minutes and shows no progress):

# cp -R /mnt/* /media/maurits/39F5-8B34/

Now unmount the ISO...

# umount /mnt

and the USB drive before removing it:

# umount /media/maurits/39F5-8B34/

To ensure the drive is OK, print the contents using 'fdisk':

# fdisk /dev/sdh

Welcome to fdisk (util-linux 2.27.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdh: 7.2 GiB, 7735541760 bytes, 15108480 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1F574274-2552-4728-831F-90D768723297

Device     Start      End  Sectors  Size Type
/dev/sdh1   2048 15108095 15106048  7.2G Microsoft basic data

Command (m for help): q

Note that the drive should say 'Disklabel type: gpt' and 'Microsoft basic data'.

Overig

Other general

[:I:M:G:SPACE:W:I:D:T:H:=:6:0:0:]:SPACE:SPACE:[:/:I:M:G:]:LEFT:LEFT:LEFT:LEFT:LEFT:LEFT:LEFT:CTRL+V:RIGHT:RIGHT:RIGHT:RIGHT:RIGHT:RIGHT:RIGHT:ENTER

Overig

Addresses

82.95.225.202 PvdM
83.119.154.28 c.s
85.150.196.180 betert
----
linux fmt 0010322941R  engeland (4802139149 op 1-12-2010)
linux mag 194754204754 duits
cchip DG7R0Y5
----
siephi a t g m a i l soepsoep
----
cbond consu l culia p 1902jp en nog een x
====
xs voip werkgever 1 exclam
====
pas aarde
11882
Js8#gfC
=====
xenpvdmattenC1tr1x01
======
tqhjwc 500142 a ke we
oggiz 10170391

========
nl postnl mijn pakket 
p 0 stp @ kk 3 tj 3
========
nav systeem. AK7HQ D2BAW
appl 9.844.634510.500 besturingssyteem 628172 NFA2.0-KZ5, 28-01-2011

128 MB ram 
gps v1.20 boot 55.7716
kaart europa v855.2885
renault.multimedia.com
==============
337 419 739 kijker groep

Overig

Repos cpu3972

#  | Alias                        | Name                                        | Enabled | Refresh | URI                                                                                              
---+------------------------------+---------------------------------------------+---------+---------+--------------------------------------------------------------------------------------------------
 1 | download.opensuse.org-webcam | openSUSE BuildService - Drivers for webcams | Yes     | Yes     | http://download.opensuse.org/repositories/drivers:/webcam/openSUSE_11.4/                         
 2 | google-talkplugin            | google-talkplugin                           | Yes     | Yes     | http://dl.google.com/linux/talkplugin/rpm/stable/x86_64                                          
 3 | packman.inode.at-suse        | Packman Repository                          | Yes     | Yes     | http://packman.inode.at/suse/11.4/                                                               
 4 | pvdm-contrib                 | pvdm-contrib                                | Yes     | Yes     | http://download.opensuse.org/repositories/openSUSE:/11.4:/Contrib/standard/                      
 5 | pvdm-fontsmoothtest          | pvdm-fontsmoothtest                         | Yes     | Yes     | http://download.opensuse.org/repositories/home:/muzlocker/openSUSE_11.4/                         
 6 | pvdm-games                   | pvdm-games                                  | Yes     | Yes     | http://download.opensuse.org/repositories/games/openSUSE_11.4                                    
 7 | pvdm-kde-core                | pvdm-kde-core                               | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Release:/47/openSUSE_11.4/                        
 8 | pvdm-kde-extra               | pvdm-kde-extra                              | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Extra/KDE_Release_47_openSUSE_11.4/               
 9 | pvdm-kde-play                | pvdm-kde-play                               | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Unstable:/Playground/KDE_Release_47_openSUSE_11.4/
10 | pvdm-kde-releasedapps        | pvdm-kde-releasedapps                       | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/UpdatedApps/openSUSE_11.4/                        
11 | pvdm-kernel                  | pvdm-kernel                                 | Yes     | Yes     | http://download.opensuse.org/repositories/Kernel:/stable/standard/                               
12 | pvdm-lo                      | pvdm-lo                                     | Yes     | No      | http://download.opensuse.org/repositories/LibreOffice:/Unstable/openSUSE_11.4/                   
13 | pvdm-soundlibs               | pvdm-soundlibs                              | Yes     | Yes     | http://download.opensuse.org/repositories/multimedia:/libs/openSUSE_11.4/                        
14 | pvdm-wine                    | pvdm-wine                                   | Yes     | Yes     | http://download.opensuse.org/repositories/Emulators:/Wine/openSUSE_11.4/                         
15 | pvdm-xorg                    | pvdm-xorg                                   | Yes     | Yes     | http://download.opensuse.org/repositories/X11:/XOrg/openSUSE_11.4/                               
16 | repo-11.4-non-oss            | openSUSE-11.4 Non-OSS                       | Yes     | No      | http://download.opensuse.org/distribution/11.4/repo/non-oss/                                     
17 | repo-11.4-oss                | openSUSE-11.4 OSS                           | Yes     | No      | http://download.opensuse.org/distribution/11.4/repo/oss/                                         
18 | repo-11.4-update             | openSUSE-11.4 Updates                       | Yes     | Yes     | http://download.opensuse.org/update/11.4/

Overig

Disks

hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
        Model Number:       WDC WD30EFRX-68EUZN0
        Serial Number:      WD-WCC4N0PFTP2N
        Firmware Revision:  82.00A82
        Transport:          Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Supported: 9 8 7 6 5
        Likely used: 9
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors: 5860533168
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     2861588 MBytes
        device size with M = 1000*1000:     3000592 MBytes (3000 GB)
        cache/buffer size  = unknown
        Nominal Media Rotation Rate: 5400
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, with device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        DMA: mdma0 mdma1 mdma2 udma0 udma1 *udma2 udma3 udma4 udma5 udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
                Media Card Pass-Through
           *    General Purpose Logging feature set
           *    64-bit World wide name
           *    URG for READ_STREAM[_DMA]_EXT
           *    URG for WRITE_STREAM[_DMA]_EXT
           *    IDLE_IMMEDIATE with UNLOAD
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Host-initiated interface power management
           *    Phy event counters
           *    Idle-Unload when NCQ is active
           *    NCQ priority information
           *    unknown 76[15]
                DMA Setup Auto-Activate optimization
                Device-initiated interface power management
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
                unknown 206[14] (vendor specific)
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
                frozen
        not     expired: security count
                supported: enhanced erase
        420min for SECURITY ERASE UNIT. 420min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee2b8f12459
        NAA             : 5
        IEEE OUI        : 0014ee
        Unique ID       : 2b8f12459
Checksum: correct

-------------------------------
hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
        Model Number:       WDC WD30EFRX-68EUZN0
        Serial Number:      WD-WCC4N6PKUR2D
        Firmware Revision:  82.00A82
        Transport:          Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Supported: 9 8 7 6 5
        Likely used: 9
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors: 5860533168
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     2861588 MBytes
        device size with M = 1000*1000:     3000592 MBytes (3000 GB)
        cache/buffer size  = unknown
        Nominal Media Rotation Rate: 5400
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, with device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        DMA: mdma0 mdma1 mdma2 udma0 udma1 *udma2 udma3 udma4 udma5 udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
                Media Card Pass-Through
           *    General Purpose Logging feature set
           *    64-bit World wide name
           *    URG for READ_STREAM[_DMA]_EXT
           *    URG for WRITE_STREAM[_DMA]_EXT
           *    IDLE_IMMEDIATE with UNLOAD
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Host-initiated interface power management
           *    Phy event counters
           *    Idle-Unload when NCQ is active
           *    NCQ priority information
           *    unknown 76[15]
                DMA Setup Auto-Activate optimization
                Device-initiated interface power management
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
                unknown 206[14] (vendor specific)
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
                frozen
        not     expired: security count
                supported: enhanced erase
        448min for SECURITY ERASE UNIT. 448min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee2b8f20a8e
        NAA             : 5
        IEEE OUI        : 0014ee
        Unique ID       : 2b8f20a8e
Checksum: correct

Overig

Repos lapx

pvdm@lapx:~> zypper lr --uri
#  | Alias               | Name                         | Enabled | Refresh | URI                                                                               
---+---------------------+------------------------------+---------+---------+-----------------------------------------------------------------------------------
 1 | google-talkplugin   | google-talkplugin            | Yes     | Yes     | http://dl.google.com/linux/talkplugin/rpm/stable/x86_64                           
 2 | packman-essentials  | packman-essentials           | Yes     | Yes     | http://packman.inode.at/suse/openSUSE_12.2/Essentials                             
 3 | packman-multimedia  | packman-multimedia           | Yes     | Yes     | http://packman.inode.at/suse/openSUSE_12.2/Multimedia                             
 4 | pvdm-fonts          | pvdm-fonts                   | Yes     | Yes     | http://download.opensuse.org/repositories/home:/muzlocker/openSUSE_12.2/          
 5 | pvdm-kde49          | pvdm-kde49                   | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Release:/49/openSUSE_12.2/         
 6 | pvdm-kde49extra     | pvdm-kde49extra              | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Extra/KDE_Release_49_openSUSE_12.2/
 7 | pvdm-kernel         | pvdm-kernel                  | Yes     | Yes     | http://download.opensuse.org/repositories/Kernel:/HEAD/standard/                  
 8 | pvdm-xorg           | pvdm-xorg                    | Yes     | Yes     | http://download.opensuse.org/repositories/X11:/XOrg/openSUSE_12.2/                
 9 | repo-non-oss        | openSUSE-12.2-Non-Oss        | Yes     | Yes     | http://download.opensuse.org/distribution/12.2/repo/non-oss/                      
10 | repo-oss            | openSUSE-12.2-Oss            | Yes     | Yes     | http://download.opensuse.org/distribution/12.2/repo/oss/                          
11 | repo-update         | openSUSE-12.2-Update         | Yes     | Yes     | http://download.opensuse.org/update/12.2/                                         
12 | repo-update-non-oss | openSUSE-12.2-Update-Non-Oss | Yes     | Yes     | http://download.opensuse.org/update/12.2-non-oss/


#  | Alias               | Name                         | Enabled | Refresh | URI                                                                                 
---+---------------------+------------------------------+---------+---------+-------------------------------------------------------------------------------------
 1 | kde412              | kde412                       | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Current/openSUSE_13.1/               
 2 | kde412extra         | kde412extra                  | Yes     | Yes     | http://download.opensuse.org/repositories/KDE:/Extra/KDE_Current_openSUSE_13.1/     
 3 | kernel              | kernel                       | No      | Yes     | http://ftp5.gwdg.de/pub/opensuse/repositories/Kernel:/HEAD/standard/                
 4 | packman             | packman                      | Yes     | Yes     | http://packman.inode.at/suse/openSUSE_13.1/                                         
 5 | pipelight           | pipelight                    | Yes     | Yes     | http://download.opensuse.org/repositories/home:/DarkPlayer:/Pipelight/openSUSE_13.1/
 6 | pvdm-publishing     | pvdm-publishing              | Yes     | Yes     | http://download.opensuse.org/repositories/Publishing/openSUSE_13.1/                 
 7 | repo-non-oss        | openSUSE-13.1-Non-Oss        | Yes     | Yes     | http://download.opensuse.org/distribution/13.1/repo/non-oss/                        
 8 | repo-oss            | openSUSE-13.1-Oss            | Yes     | Yes     | http://download.opensuse.org/distribution/13.1/repo/oss/                            
 9 | repo-update         | openSUSE-13.1-Update         | Yes     | Yes     | http://download.opensuse.org/update/13.1/                                           
10 | repo-update-non-oss | openSUSE-13.1-Update-Non-Oss | Yes     | Yes     | http://download.opensuse.org/update/13.1-non-oss/

Overig

Smt

[NU]
NUUrl = https://nu.novell.com/
NURegUrl = https://secure-www.novell.com/center/regsvc/
NUUser = XXXXXXXXXXXXXXXXXXXX
NUPass = XXXXXXXXXXXXXXXXXXX

[DB]
config = dbi:mysql:database=smt;host=localhost
user = smt
pass = XXXXXXXXXXXXXXXXXXXXXX

[LOCAL]
# Default should be http://server.domain.top/
url = XXXXXXXXXXXXXXXXXXXXX
# This email address is used for registration at NCC
nccEmail = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
MirrorTo = /data/smt
MirrorAll = false
MirrorSRC = false
forwardRegistration = true
rndRegister = 331
# specify proxy settings here, if you do not want to use the global proxy settings
# If you leave these options empty the global options are used.
#
# specify which proxy you want to use for HTTP connection
# in the form http://proxy.example.com:3128
HTTPProxy =
# specify which proxy you want to use for HTTPS connection
# in the form http://proxy.example.com:3128
HTTPSProxy =
# specify username and password if your proxy requires authentication
# in the form username:password
ProxyUser =
#
# require authentication to access the repository?
# Three possible authtypes can be configures here
# 1) none   : no authentication required (default)
# 2) lazy   : check only username and password. A valid user has access to all repositories
# 3) strict : check also if this user has access to the repository.
#
requiredAuthType = none
#
# the smt commands should run with this unix user
#
smtUser = smt
#
# ID of the GPG key to be used to sign modified (filtered) repositories.
# The key must be accessible by the user who runs SMT, i.e. the user specified
# in the 'smtUser' configuration option.
#
# If empty, the modified repositories will be unsigned.
#
signingKeyID =

[REST]
# Enable administrative access to the SMT RESTService by setting enableRESTAdminAccess=1
# default: 0
enableRESTAdminAccess = 0
# Define the username the REST-Admin uses for login
# default: RESTroot
RESTAdminUser = RESTroot
# Define the password for the REST-Admin (note: empty password is invalid)
# default: <empty>
RESTAdminPassword =

[JOBQUEUE]
# maximum age of finished (non-persistent) jobs in days
# default: 8
maxFinishedJobAge = 8

[REPORT]
# comma separated list of eMail addresses where the status reports will be sent to
reportEmail = XXXXXXXXXXXXXXXXXXXXXX
# from field of report mails - if empty it defaults to "root@<hostname>.<domainname>"
reportEmailFrom =
# relay mail server - leave emtpy if mail should be sent directly
mailServer =
mailServerPort =
# mail server authentication - leave empty if not required
mailServerUser =
mailServerPassword =

Overig

Certificate renewal apache

# zypper in python3-certbot-apache
# certbot --authenticator standalone --installer apache -d www.reeltoreel.nl --pre-hook "systemctl stop apache" --post-hook "systemctl start apache"

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer apache
Running pre-hook command: systemctl stop apache
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.reeltoreel.nl
Waiting for verification...
Cleaning up challenges
Running post-hook command: systemctl start apache
Deploying Certificate to VirtualHost /etc/apache2/vhosts.d/reeltoreel-le-ssl.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Enhancement redirect was already set.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled https://www.reeltoreel.nl

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=www.reeltoreel.nl
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   Your key file has been saved at:
   Your cert will expire on 2019-06-20. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Overig

Athana PF-212 Bulk Eraser

Athana International - PF212 Degausser - Model PF 212

GENEVA-Audio-Video-Bulk-Tape-Eraser-Model-PF-212.jpg|Model PF 212

Computer/Video/Audio Tape Eraser

(220 VAC; 50 Hz)

The hand-held model PF 212 tape eraser is perfect for the low-volume user. A few seconds is all it takes to completely erase a tape. This convenient, easy-to-use machine will completely erase previous recordings in just seconds. Used computer tapes and diskettes can perform like new when erased with a wave of the PF 212 professional, powerful tape eraser. At 220 VAC, this machine is powered for European and Asian markets.

The duty cycle is one minute on and five minutes off.

Features:

2300 gauss flux field erases tapes to like-new condition
Thermal fuse prevents overheating; resets automatically
Easy to activate on/off power switch
Rugged, comfort-molded ABS Cycolac case
UL/CSA approved
220 VAC; 50/60 Hz power required

Applications:

Police recordings
Fire department recordings
TV broadcast recordings
Ambulance training recordings
Hospital recordings
Bank data processing records
Financial and insurance recordings
Software duplicators
Military video recordings
Legal confidential diskette recordings
Floppy disk recordings
Video recordings
Audio recordings
TV screens
CRT terminal screens
Music industry audio recordings
Data processing obselete records

Media Erased:

Diskettes: 3 1/2" and 5 1/4" diskettes, floppy disks
Audio tape cassettes
10 1/2" computer reels
Reel tape: 1/4" and 1/2" audio reels
IBM 3480 and 3490 cartridges
TK50, TK70 and TK75 tape cartridges
DC 300, 600, 1000 and 2000 cartridges
VHS and S-VHS videotape cartridges
U-Matic cartridges
Betacam videotape cartridges
Compact cassettes

 

Purchase Price  $169.00

Shipping    $9.95

Gauss   2300

Dimensions

Width 5.5 inches 140 mm 

Length 10.25 inches 260 mm 

Height 3.75 inches 95 mm

Weight 3 lbs 1.4 kg

Overig

Fix bash history

put in .bashrc or bash.bashrc.local

# fix bash history
export HISTCONTROL=ignoredups:erasedups  # no duplicate entries
export HISTSIZE=100000                   # big big history
export HISTFILESIZE=100000               # big big history
shopt -s histappend                      # append to history, don't overwrite it

# Save and reload the history after each command finishes
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"
# end fix bash

Overig

Determine which vga driver is in use

lapx: # lspci -nnk |grep -A3 VGA
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Thames [Radeon HD 7550M/7570M/7650M] [1002:6841]
        DeviceName: 0
        Subsystem: Hewlett-Packard Company Device [103c:17a9]
        Kernel driver in use: radeon

lesie:~ # lspci -nnk |grep -A3 VGA
00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 620 [8086:5917] (rev 07)
        Subsystem: Lenovo Device [17aa:396b]
        Kernel driver in use: i915
        Kernel modules: i915

amd64:~ # lspci -nnk |grep -A3 VGA
00:01.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Richland [Radeon HD 8570D] [1002:990e]
        Subsystem: ASRock Incorporation Device [1849:990e]
        Kernel driver in use: radeon
        Kernel modules: radeon

Overig

Envy

HP Envy x360 Convertible

Model 15-ed0300nd

ProdID 3K787EA#ABH ---

Productnummer	3K787EA
Productnaam	HP ENVY x360 Laptop - 15-ed0300nd
Microprocessor	Intel® Core™ i7-10510U (1,8 GHz basisfrequentie, tot 4,9 GHz basisfrequentie met Intel® Turbo Boost-technologie, 8 MB cache, 4 cores)
Chipset	Intel® geïntegreerde SoC
Geheugen, standaard	16 GB DDR4-2666 SDRAM (2 x 8 GB)
Videographics	NVIDIA® GeForce® MX330 (4 GB GDDR5 gereserveerd)
Vaste schijf	1 TB PCIe® NVMe™ M.2 SSD
Optische drive	Optische drive niet inbegrepen
Beeldscherm	39,6 cm (15,6 inch) wled-backlit FHD IPS met smalle randen, glas van rand tot rand en multitouch-ondersteuning, 250 nits, 45% NTSC (1920 x 1080)
Draadloze verbinding	Intel® Wi-Fi 6 AX 201 (2x2) en Bluetooth® 5 combo (ondersteuning voor Gigabit-overdrachtssnelheden)
Uitbreidingssleuven	1 multi-formaat SD-mediakaartlezer
Externe poorten	1 SuperSpeed USB Type-C® 10Gbps signaling rate (USB Power Delivery, DisplayPort™ 1.2, HP Sleep and Charge); 1 SuperSpeed USB Type-A 5Gbps signaling rate (HP Sleep and Charge); 1 SuperSpeed USB Type-A 5Gbps signaling rate; 1 HDMI 2.0; 1 AC smart pin; 1 headphone/microphone combo
Minimumafmetingen (b x d x h)	35,8 x 23 x 1,89 cm
Gewicht	1,97 kg
Voedingstype	65-Watt netvoedingsadapter
Batterijtype	3-cels, 51-WHr lithium-ion
Accuwerktijd bij gemengd gebruik	Tot 8 uur en 30 minuten
Accuwerktijd bij afspelen van video's	Tot 10 uur en 45 minuten
Webcam	HP Wide Vision HD-camera met cameraklepje en ingebouwde digitale dual array-microfoon
Audiokenmerken	Audio van Bang & Olufsen; 3 speakers; HP Audio Boost 2.0

Envy:~ # inxi -F
System:
  Host: Envy Kernel: 5.19.8-1-default arch: x86_64 bits: 64
    Console: pty pts/1 Distro: openSUSE Tumbleweed 20220914
Machine:
  Type: Convertible System: HP product: HP ENVY x360 Convertible 15-ed0xxx
    v: Type1ProductConfigId serial: CND02035YM
  Mobo: HP model: 8757 v: 08.32 serial: PJUJAC21TDPB4P UEFI: Insyde v: F.11
    date: 06/08/2020
Battery:
  ID-1: BAT1 charge: 40.2 Wh (100.0%) condition: 40.2/51.0 Wh (78.9%)
CPU:
  Info: quad core model: Intel Core i7-10510U bits: 64 type: MT MCP cache:
    L2: 1024 KiB
  Speed (MHz): avg: 2300 min/max: 400/4900 cores: 1: 2300 2: 2300 3: 2300
    4: 2300 5: 2300 6: 2300 7: 2300 8: 2300
Graphics:
  Device-1: Intel CometLake-U GT2 [UHD Graphics] driver: i915 v: kernel
  Device-2: NVIDIA GP108M [GeForce MX330] driver: N/A
  Device-3: Chicony HP Wide Vision HD Camera type: USB driver: uvcvideo
  Display: server: X.org v: 1.21.1.4 with: Xwayland v: 22.1.3 driver: X:
    loaded: modesetting,nvidia gpu: i915 resolution: 1920x1080~60Hz
  OpenGL: renderer: Mesa Intel UHD Graphics (CML GT2) v: 4.6 Mesa 22.1.7
Audio:
  Device-1: Intel Comet Lake PCH-LP cAVS driver: sof-audio-pci-intel-cnl
  Sound Server-1: ALSA v: k5.19.8-1-default running: yes
  Sound Server-2: PulseAudio v: 16.1 running: yes
  Sound Server-3: PipeWire v: 0.3.57 running: yes
Network:
  Device-1: Intel Comet Lake PCH-LP CNVi WiFi driver: iwlwifi
  IF: wlo1 state: up mac: 3c:58:c2:7c:f3:d5
Bluetooth:
  Device-1: Intel AX201 Bluetooth type: USB driver: btusb
  Report: rfkill ID: hci0 state: up address: see --recommends
RAID:
  Hardware-1: Intel 82801 Mobile SATA Controller [RAID mode] driver: ahci
Drives:
  Local Storage: total: 953.87 GiB used: 570.99 GiB (59.9%)
  ID-1: /dev/nvme0n1 vendor: Intel model: SSDPEKNW010T8H size: 953.87 GiB
Partition:
  ID-1: / size: 100 GiB used: 38.44 GiB (38.4%) fs: btrfs dev: /dev/nvme0n1p5
  ID-2: /boot/efi size: 256 MiB used: 81 MiB (31.7%) fs: vfat
    dev: /dev/nvme0n1p1
  ID-3: /home size: 100 GiB used: 61.38 GiB (61.4%) fs: btrfs
    dev: /dev/nvme0n1p6
  ID-4: /opt size: 100 GiB used: 38.44 GiB (38.4%) fs: btrfs
    dev: /dev/nvme0n1p5
  ID-5: /var size: 100 GiB used: 38.44 GiB (38.4%) fs: btrfs
    dev: /dev/nvme0n1p5
Swap:
  ID-1: swap-1 type: partition size: 15.37 GiB used: 0 KiB (0.0%)
    dev: /dev/nvme0n1p8
Sensors:
  System Temperatures: cpu: 38.0 C pch: 35.0 C mobo: N/A
  Fan Speeds (RPM): N/A
Info:
  Processes: 359 Uptime: 0h 58m Memory: 15.37 GiB used: 2.67 GiB (17.4%)
  Shell: Bash inxi: 3.3.21

Overig

Set cpu performance

# cpupower frequency-set -g performance

#!/bin/sh
#
# puts cpus in performance mode
#
echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

Overig

Xorg

xrandr

Overig

Xrandr

XRandR

--query -> details about grapics adapter
--output LVDS|LCD|VGA|HDMI --mode 800x600 -> change resolution
--output LVDS|LCD|VGA|HDMI --auto -> restore resolution
--output LVDS|LCD|VGA|HDMI --rate 60

connecting a projector

clone VGA and LVDS outputs:

xrandr --output VGA --mode 1024x768 --same-as LVDS --output LVDS --mode 1024x768

switch off projector:

xrandr --output VGA --off
xrandr --output LVDS --auto

switch off internal display and use external display:

xrandr --output LVDS --off --output VGA --auto

multi-head

xrandr --output LVDS --auto --left-of VGA --output VGA --auto

rotation

xrandr --output LVDS --rotate left|right|normal|invert

Overig

Clear recently used documents permanently in KDE4

To do this:

solution: type in terminal

rm ~/.kde4/share/apps/RecentDocuments/*
chmod 000 ~/.kde4/share/apps/RecentDocuments

The first line will clear the directory where the shortcuts to the recent files would be stored

The second line will make the directory where the shortcuts to the recent files would be stored, non-writable, so no history is recorded...

Overig

How to send mail via telnet to a server using SMTP commands

Sending an e-mail via Telnet

In this document, we:

shall use Telnet to connect to the SMTP server.
shall give commands to the server, then type our e-mail, and finally tell the server, 'Okay, the e-mail is done. Send it.
can then send more mails, or disconnect from the server.

Connect

telnet mail.monitortools.com 25

This will open a Telnet window, and within a short time, you will be connected to the SMTP server, and the server says:

220 PROTAGONISTNT Mailmax version 4. 8. 3. 0 ESMTP Mail Server Ready

This varies, but you should definitely see the '220' part. It is an indication that the server is ready to service your request.

Handshake

Now the server expects you to identify yourself. You can enter the name of your computer or anything else you want. If you have a domain-name, then you should enter the domain-name here. My computer's name is dell01, so I say:

helo dell01

Note that it is 'helo' and not 'hello'. The commands are not case-sensitive, so you can also say HeLo or HELO or hELo. The server replies:

250 HELO 217.120.215.201, How can I help you?

This is like a shake-hand. You tell the server your name, and it says its name! Computers are quite friendly, you see!

From

Next give the server your e-mail address. Note that most SMTP servers require that your e-mail address belong to the same domain as the server. For example, if you send mail from Yahoo! SMTP server, you should have a Yahoo! address. You cannot use it if you give it a Hotmail address. Let me give the SMTP server some e-mail address:

mail from: webmaster@monitortools.com

'mail from:' is a SMTP command. Note that there is a space between 'mail' and 'from', followed by a colon (:). The server says:

250 Ok

To

Tell the server who you want to send the e-mail to. Let me send a mail to info@activexperts:

rcpt to: info@activexperts.com

There are no restrictions here. You can enter any e-mail address. If there is some problem with the recipient-address, your mail will bounce, but for now, the server doesn't complain. It will say:

250 Ok

Body

You have told the server your e-mail address, and the recipient's e-mail address, so now you can go ahead and type the e-mail. You have to do that with the data command:

data

The server asks you to go ahead with your e-mail:

354 End data with .

Send

Now type in your e-mail, like this:

This is a test e-mail.
Remember to type it all right. Backspace key doesn't work in Windows
Telnet, though it does in Linux. If you make a mistake, try pressing
CTRL-h. If it works, well and good.

When you finish your e-mail, press [ENTER], then a '.', and again an [ENTER]. This tells the server that you have finished the e-mail, and it can send it. It will say:

250 Ok: queued as 6AB5150038

Your mail was sent!

Disconnect

Now you can either send another mail, or disconnect from the server. If you want to send another mail, you should repeat the 'rcpt to:' and 'data' commands. There is no need for 'helo' and 'mail from:', because the server already knows who you are. If you want to disconnect, just say 'quit':

quit

The server will reply:

221 Bye

and you will lose connection with the server.

Overig

How to toggle compositing in KDE on the CLI command line

> qdbus org.kde.kwin /KWin org.kde.KWin.toggleCompositing

Overig

New Page

					De-Equalizer	Digital Drive Signal Processor	Error Correction Codec	Tape Format and Error Correction?	Drive Processor	Subband Codec	Subband Filter	Stereo Filter and Codec	Adaptive Allocation and Scaling	Digital Audio Interface		Digital to Analog Converter	Analog to Digital Converter
Device	Head (A+P+R)	Bits	Read	Write	DEQ *2	DDSP	ERCO	TFE	DRP/SPD1	SBC	SBF	SFC	ADAS	DAI/DAIO	Dig Filter	DAC	ADC	ADC/DAC	Dolby
DCC900	2+9+9	16	TDA1317	TDA1316T/N-T	SAA2051	SAA2041	SAA2031	n/a	n/a	SAA2021	2x SAA2001	n/a	SAA2011	M51581FS	SM5840FS	SAA7350	AK5326	n/a	CXA1331
RS-DC10	2+9+9	16	TDA1317	TDA1316T	SAA2051	SAA2041	SAA2031	n/a	n/a	SAA2021	2x SAA2001	n/a	SAA2011	M51581FP	SM5840FS	SAA7350	AK5326	n/a	CXA1331
DD82/92	2+9+9	16	TDA1317	TDA1316T	SAA2051	SAA2041	SAA2031	n/a	n/a	SAA2021	2x SAA2001	n/a	SAA2011	M51581FP	SM5840FS	SAA7350+TDA1547 *da	AK5326	n/a	CXA1331
DCC600	2+9+9	16	TDA1317	TDA1316T	SAA2051	SAA2041	SAA2031	n/a	n/a	SAA2021	2x SAA2001	n/a	SAA2011	M51581P	*df	SAA7321	AK5339	n/a	CXA1331
DCC300/380	2+9+9	16	TDA1317	TDA1316T	SAA2051	SAA2041	SAA2031	n/a	n/a	SAA2021	2x SAA2001	n/a	SAA2011	M51581FP	*df	SAA7323	AK5339	n/a	CXA1331?
DCC91/450	2+9+9	16	TDA1317	TDA1316T	SAA2051	SAA2041	SAA2031	n/a	n/a	SAA2021	2xSAA2001	n/a	SAA2011	M51581FP	*df	SAA7323	AK5339	n/a	CXA1331
DCC130	0+18+0	16	2xTDA1318H	*p	SAA2032GP	*t	*t	SAA2022GP	n/a	*f	*f	SAA2002GP	*p	M51581GP	SM5881SET	UPD63200GSE2	*p	n/a	BA1106FST2
DCC134	0+18+0	18	TDA1380	*p	*d	*d	*d	n/a	SAA3323	*f	*f	SAA2003	*p	p i	SM5881S	TDA1313T	*p	n/a	BA1106FS
DCC170	4+18+18	18	TDA1380	TDA1381	*d	*d	*d	n/a	SAA3323	*f	*f	SAA2003	SAA2013	TDA1315	*df	*ad	*ad	TDA1309H	BA1106FS
DCC175	4+18+18	18	TDA1380	TDA1381	*d	*d	*d	n/a	SAA3323	*f	*f	SAA2003	SAA2013	TDA1315	*df	*ad	*ad	TDA1309H	BA1106FS
DCC730	2+9+9	18	TDA1317	TDA1319	*d	*d	*d	n/a	SAA3023	*f	*f	SAA2003	SAA2013	TDA1315	*df	TDA1305T	SAA7366	n/a	CXA1331S
DCC951	2+9+9	18	TDA1317	TDA1319	*d	*d	*d	n/a	SAA3023	*f	*f	SAA2003	SAA2013	TDA1315	*df	TDA1305T	SAA7366	n/a	CXA1331S
RS-DC8	2+9+9	?	AN7631FHPV	AN7639SBE2V	*s	*s	*s	n/a	MN66703	MN66702	MN66701	n/a	*m	M65812FP	*df	MN64782	MN6463	n/a	AN7374S-E2

Legend

*p	Not needed, this device is a player only
*d	Functionality integrated in Drive Processor
*t	Functionality integrated in TFE
*s	Matsushita integrated functionality into "SPDI"
*m	Matsushita integrated functionality into "SBC"
*f	SBC and SBF replaced by SFC?
*i	No digital I/O, analog I/O only
*df	Digital Filter not needed for DAC in this device
*ad	ADC and DAC integrated into one chip
*da	The TDA1547 works together with the SAA7350

Chip Index

AK5326	Analog Digital Converter
AK5339	Analog Digital Converter
AN7374S-E2	Dolby Playback Processor
AN7631FHPV	Read Amplifier
AN7639SBE2V	Write Amplifier
BA1106FS	Dolby Playback Processor
BA1106FST2	Dolby Playback Processor
CXA1331	Dolby Playback Processor
CXA1331S	Dolby Playback Processor
M51581FP	Digital Audio Interface
M51581FS	Digital Audio Interface
M51581GP	Digital Audio Interface
M51581P	Digital Audio Interface
M65812FP	Digital Audio Interface
MN6463	Analog Digital Converter
MN64782	Digital Analog Converter
MN66701	Subband Filter
MN66702	Subband Codec
MN66703	Speed Controller
SAA2001	Subband Filter
SAA2002GP	Subband Filter and Codec
SAA2003	Stereo Filter and Codec
SAA2011	Adaptive Allocation and Scaling
SAA2013	Adaptive Allocation and Scaling
SAA2021	Subband Codec
SAA2022	Tape Format and Error Correction
SAA2031	Error Correction Codec
SAA2032GP	De-equalizer
SAA2041	Digital Drive Signal Processor
SAA2051	De-equalizer
SAA3023	Drive Processor
SAA3323	Drive Processor
SAA7321	Digital Analog Converter
SAA7323	Digital Analog Converter
SAA7350	Digital Analog Converter
SAA7366	Analog Digital Converter
SM5840FS	Digital Filter
SM5881S	Digital Filter
SM5881SET	Digital Filter
TDA1305T	Digital Analog Converter
TDA1309H	ADC/DAC
TDA1313T	Digital Analog Converter
TDA1315	Digital Audio Interface
TDA1316T	Write Amplifier
TDA1317	Read Amplifier
TDA1318H	Read Amplifier
TDA1319	Write Amplifier
TDA1380	Read Amplifier
TDA1381	Write Amplifier
TDA1547	Digital Analog Converter
UPD63200GSE2	Digital Analog Converter

Webcams & Devices

Android

MyBackup - to backup your files like just before an upgrade!

Webcams & Devices

Philips Webcam

vcnc

Technical specifications

Optical
PCVC740K	ToUcam Pro
Sensor	CCD
Pixels	640 (H) x 480 (V)
Still image resolution	1280 (H) x 960 (V)
Illumination	< 1 lux
Integrated lens	F2.0

Resolution/performance
Output resolution	Pixels (H x V)	Frame rate in frames/sec [fps]
VGA	640 x 480	Up to 30
CIF	352 x 288	Up to 30
SIF	320 x 240	Up to 60
QCIF	176 x 144	Up to 60
QSIF	160 x 120	Up to 60

Data format	I420, IYUV

Webcams & Devices

Install Dymo Labelwriter in openSUSE

To install the Dymo Labelwriter 450 (and probably others as well) in openSUSE 13.2 (and probably others as well) do the following:

#zypper in cups-devel

Download the drivers from the dymo site from here

Unpack to a directory, in my case dymo-cups-drivers-1.4.0.5/ and open a terminal in that dir. Then:

#./configure
#make

Then goto http://localhost:631 and you can add the printer and configure it as normal.

Have fun!

Webcams & Devices

Detect webcam properties

# for d in /dev/video* ; do echo $d ; v4l2-ctl --device=$d -D --list-formats ; echo '==============='; done

# v4l-info /dev/video0

Webcams & Devices

Logitech quickcam

pvdm@amd64:zm > v4l-info /dev/video0

### v4l2 device info [/dev/video0] ###
general info
    VIDIOC_QUERYCAP
        driver                  : "pwc"
        card                    : "Logitech QuickCam Zoom"
        bus_info                : "usb-0000:00:10.1-2.2"
        version                 : 4.4.87
        capabilities            : 0x85200001 [VIDEO_CAPTURE,?,READWRITE,STREAMING,(null)]

standards

inputs
    VIDIOC_ENUMINPUT(0)
        index                   : 0
        name                    : "Camera"
        type                    : CAMERA
        audioset                : 0
        tuner                   : 0
        std                     : 0x0 []
        status                  : 0x0 []

video capture
    VIDIOC_ENUM_FMT(0,VIDEO_CAPTURE)
        index                   : 0
        type                    : VIDEO_CAPTURE
        flags                   : 1
        description             : "Raw Philips Webcam Type (New)"
        pixelformat             : 0x32435750 [PWC2]
    VIDIOC_ENUM_FMT(1,VIDEO_CAPTURE)
        index                   : 1
        type                    : VIDEO_CAPTURE
        flags                   : 0
        description             : "Planar YUV 4:2:0"
        pixelformat             : 0x32315559 [YU12]
    VIDIOC_G_FMT(VIDEO_CAPTURE)
        type                    : VIDEO_CAPTURE
        fmt.pix.width           : 160
        fmt.pix.height          : 120
        fmt.pix.pixelformat     : 0x32315559 [YU12]
        fmt.pix.field           : NONE
        fmt.pix.bytesperline    : 160
        fmt.pix.sizeimage       : 28800
        fmt.pix.colorspace      : SRGB
        fmt.pix.priv            : 4276996862

Zoneminder

PAL M, YUV420, 24bit color, 320x240

Webcams & Devices

Gspca

pvdm@amd64:zm > v4l-info /dev/video2

### v4l2 device info [/dev/video2] ###
general info
    VIDIOC_QUERYCAP
        driver                  : "pac207"
        card                    : "CIF Single Chip     "
        bus_info                : "usb-0000:00:10.1-2.4"
        version                 : 4.4.87
        capabilities            : 0x85200001 [VIDEO_CAPTURE,?,READWRITE,STREAMING,(null)]

standards

inputs
    VIDIOC_ENUMINPUT(0)
        index                   : 0
        name                    : "pac207"
        type                    : CAMERA
        audioset                : 0
        tuner                   : 0
        std                     : 0x0 []
        status                  : 0x0 []

video capture
    VIDIOC_ENUM_FMT(0,VIDEO_CAPTURE)
        index                   : 0
        type                    : VIDEO_CAPTURE
        flags                   : 1
        description             : "GSPCA PAC207"
        pixelformat             : 0x37303250 [P207]
    VIDIOC_G_FMT(VIDEO_CAPTURE)
        type                    : VIDEO_CAPTURE
        fmt.pix.width           : 352
        fmt.pix.height          : 288
        fmt.pix.pixelformat     : 0x37303250 [P207]
        fmt.pix.field           : NONE
        fmt.pix.bytesperline    : 352
        fmt.pix.sizeimage       : 101952
        fmt.pix.colorspace      : SRGB
        fmt.pix.priv            : 4276996862

Webcams & Devices

Webcams

Webcams & Devices

Creative cylinder webcam

===============
/dev/video1
Driver Info (not using libv4l2):
        Driver name   : gspca_zc3xx
        Card type     : USB Camera (041e:401f)
        Bus info      : usb-0000:00:10.1-2.1
        Driver version: 4.4.87
        Capabilities  : 0x85200001
                Video Capture
                Read/Write
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x05200001
                Video Capture
                Read/Write
                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'JPEG' (compressed)
        Name        : JFIF JPEG

===============

Zoneminder:

PAL,auto, 24bit color, 320x240

Webcams & Devices

P2pwificam

Dit is de 1e webcam

Oftewel ICAM-608 IHOMECAM

MAC: 00:ED:AC:4F:BF:B9

http://192.168.1.30:8888/index.htm

http://192.168.1.30:8888/jpeg.html

http://192.168.1.30:8888/videostream.cgi

http://192.168.1.30:8888/videostream.cgi?user=admin&pwd=

Zoneminder:

HTTP, simple, IP, port, Remote Host Path:/videostream.cgi?user=admin&pwd= , 640x360

HTTP, Regexp, 192.168.1.30, 8888, /videostream.cgi?user=admin&pwd=admin, 24 bit color, 640x360

camera1.conf

cat camera1.conf
#PVDM

camera_id = icam608
lext_left CAMERA1
width 1280
height 720
netcam_url http://192.168.1.30:8888/videostream.cgi?user=admin&pwd=
netcam_keepalive on
target_dir /data/motion/
picture_filename CAM1-%q

Webcams & Devices

Usb wifi dongles

Edup: (steekt iets uit)

Chipset:RTL8192CU
Wireless Protocol:802.11a/g,802.11n
Transmission rate:300mbps
Interface Type:USB2.0
EDUP EP-N1557 300Mbps mini 2.4GHz wifi adapter support windows and mac iOS 10.6~10.12

Note:

Please install driver successfully before plug wireless adapter into your computer

Please download the driver file here:http://www.szedup.com/support/driver-download/ep-n1557-driver/,if your desktop computer or laptop doesn't have CD rom.

Specifications:

Chipset RTL8192CU

Standard IEEE802.11 a/b/g/n

USB2.0 

Built-in 1dBi antennas

Frequency 2.4GHz

Transmission speed 2.4G up to 300Mbps

Support  soft AP 

Operation system windows XP / Vista / 7 / 8 / 8.1 / 10 ,MAC OS X 10.12.X /10.11.X / 10.10.X / 10.9.X / 10.8.X /10.7.X/10.6.X/10.5.X/10.4.X

Package content:

1*300Mbps wireless adapter

1*Driver CD

Webcams & Devices

Tethering or how to use your android as an internet modem

Did you know that you can use your android mobile phone as an internet modem?

This can be handy when you are in a spot where there is no internet connectivity through wifi, for instance in a hotel.

Just follow these simple steps:

Install Proxoid from the marketplace
On your android go to Menu - Settings/Instellingen - Applications/Toepassingen - Development/Ontwikkeling and enable USB debugging/USB foutopsporing
As root on your pc, connect your phone with USB, type 'lsusb' and note the ID number for your phone. There will be two 4-digit numbers
Again as root, create a file called '/etc/udev/rules.d/11-android.rules' and fill it as follows:

SUBSYSTEM=='usb',
SYSFS{idVendor}=="0bb4",
SYSFS{idProduct}=="00ff9",
MODE="0666",OWNER="yourname"

replacing the 2 numbers and your username

Install the Android SDK. Preferably through your packagemanager, or download from 'developer.android.com/sdk'. Extract the tarball to your homedir
Reconnect your phone.
Then, goto the 'andriod-sdk-linux_86/tools' directory and run './adb forward tcp:8080 tcp:8080'.
Start Proxoid on your android
Configure your browser to access the internet through a proxy called 'localhost' on port 8080 and your done!

If you want to stop using your phone, simply turn off the proxy in your browser

openSUSE

OpenSUSE specific

openSUSE

Speed up tips for (suse) linux installation

Speed up Linux installation

Tip: Speeding up the install time on SUSE Linux Applies to:

   * Novell Linux Desktop 9
   * Open Enterprise Server
   * SUSE Linux Professional 9.3
   * SUSE Linux 10
   * SUSE Linux Enterprise Server

SYNOPSIS: hdparm - get/set hard disk parameters

DESCRIPTION:

This cool solution will show you how it is possible to speed up the install time of your CD-ROM installation of SUSE Linux by using Linux shell command hdparm. hdparm is an a command line interface to various ATA/IDE hard disk device drivers.1 Warning, 'hdparm' may force your system to use higher values without regards to your hardware's limitations. It is a risk to use this program improperly and may cause damage to your systems. Please use extreme caution and do not exceed your hardwares limitations. Additionally WE TAKE NO RESPONSIBILITY FOR DAMAGED HARDWARE OR CORRUPTED DATA.

Know bugs are listed on Source Forge's website see notes for information

The following steps are needed to use hdparm in the installation of your SuSE LINUX system. Please note that this command only works with IDE devices, CDROMS and Hard-drives.

Place the SUSE Linux disk 1; the install cd in the CD-ROM and boot to that drive.
On the first screen enter desired parameters you need to perform your specific SuSE LINUX installation.
The next screen is the Licensing Screen; on this screen select the "I Agree" button.
Next is the Language screen; on this screen choose your language and click "Accept".
This is the bit we have been waiting for: This next screen is Installation Setting screen.
Here we will drop into a shell prompt by using Ctrl+Alt+F2. This key combination will normally take you out of your Xwindow session and drop you into a terminal (tty2). Since we are in the process of installing SUSE LINUX, this key combination will take us away from the installation screen and drop you in to a terminal (tty2), or command prompt. Don't worry. Your installation is still there. You can return by using the Alt+F7.
Once on the command line type:

hdparm /dev/hda thenENTER

You will see something similar to this.

/dev/hda:
multcount = 16 (on)
IO_support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 0 (off)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 9726/255/63, sectors = 156250000, start =34523

Notice the multcount is set to 16, using_dma is set to off, and that the IO Support is set to 16 bit.
Re-type the command:

hdparm -c1 -d1 -m1 /dev/hda ENTER

Comparing the differences from the last time, you should now see that mulcount is set to 1, or on, IO support is now 32 bit and.using_dma is turned on.

/dev/hda:
multcount = 1 (on)
IO_support = 1 (32-bit)
unmaskirq = 0 (off)
using_dma = 1 (on)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 9726/255/63, sectors = 156250000, start = 0

When you complete your desired changes to your devices, you can return to the installation screen by using the Alt+F7.

muticount -m is used to get or set the sector count for a device that supports multiple sector IO.

IO support -s is used to set the IO setting on a device. Most devices are set to use 16bit. By setting this to 1 it turns on 32bit IO for faster transfers of information.

Using_dma -a by default this is turned off. DMA or Direct Memory Access is way to control system memory without using the CPU.

If you would like more information about hdparm; browse the official hdparm website at http://sourceforge.net/projects/hdparm/ or additional options may be found in the man pages. You now have the knowledge to decrease your installation times by exerting control over your IDE devices with the hdparm Linux shell command. If you have any questions about this Cool Solution, feel free to drop me an email anytime at tvickers@idltechnology.comDit e-mailadres wordt beschermd tegen spam bots, u heeft Javascript nodig om het te bekijken .

ABOUT THE AUTHOR:

Tony Vickers is the CTO and founder of IDL technology Group, Inc. IDL has been helping companies implement Enterprise Open Source Technologies since 1994. For more information, please visit us on the web located at www.idltechnology.com

Notes

1. Lord, Mark and Leppikangas, Tomi. "hdparm(8)." (Linux man page, 2005), 1.

Barr, Joe. "Hdparm and the Zen of data transfer Monday December," Enterprise Linux on the web, 07 October 2005.

<http://enterprise.linux.com/article.pl?sid=04/12/14/166253&tid=89> (December 20, 2004).

Flickenger, Rob. "Speeding up Linux Using hdparm." Linux Dev Center on the web, 07 October 2005.

<http://www.linuxdevcenter.com/pub/a/linux/2000/06/29/hdparm.html> (June 29, 2000).

Lord, Mark. "hdparm(8)." <http://sourceforge.net/projects/hdparm/ hdparm> (April, 2005).

KNOWN BUGS:

1.

ATA security options does not always work, error message: Problem issuing security command: Cannot allocate memory (http://sourceforge.net/tracker/?atid=736682&group_id=136732&func=browse).
2.

hdparm -t with 2 disks: WD (hdb) and Seagate(hda). hdparm -t /dev/hda -Timing buffered disk reads: -1504 MB in xxx seconds = MB/sec an infinite value (http://sourceforge.net/tracker/?atid=736682&group_id=136732&func=browse).

1.

When making 6.0 or 6.1 an "987: IDE_DRIVE_TASK_OUT undeclared" error occur(http://sourceforge.net/tracker/?atid=736682&group_id=136732&func=browse).
2.

Acoustic management for SAMSUNG SP1614N disk the -M option to control the acoustic level, but it seems non-functional (http://sourceforge.net/tracker/?atid=736682&group_id=136732&func=browse).

openSUSE

Problem w/ ZEN/rug/rpm database

'''Problem w/ ZEN/rug/rpm database

this problem is obsolete, it happened with OpenSUSE 10.2 and is no longer an issue in 10.3

Things to try when your openSUSE gets errors with the package database:

rm -rf /var/tmp/* && rczmd restart && rug refresh

works around the problem. I'm not sure how to reproduce it though. It seems to happen randomly.

openSUSE

Speedup Package Management

Speeding up the package manager

Package Management is one part of the system where massive databases are thrown around, changed incrementally and is generally used heavily. Due to incremental nature of modifications it will cause the databases used to fragment, both internally (in the database blob) and externally (on the disk).

In openSUSE 10.3 we have two kinds of databases for package management.

The zypp cache SQLITE database in /var/cache/zypp/zypp.db.
The RPM database in /var/lib/rpm/.

Defragment internally

Cleaning up sqlite databases can be done using the VACUUM command.

: For the ZYPP cache database:; sqlite3 /var/cache/zypp/zypp.db vacuum; Or alternatively regenerate it from scratch:; rm /var/cache/zypp/zypp.db ; zypper refresh

Defragmenting the RPM database

: After largish update it is helpful to run rpm --rebuilddb once.

: This is just necessary after lots of packages changed/updated/installed and takes some time (10 minutes on a Laptop).

Defragment externally

While Linux filesystems try to not create heavily fragmented files, the slow increasing nature of the databases used in packagemanagement has fragmentation patterns.

So to best way to get rid of it, is to recreate the files using the pattern:

cp file file.new  # check for errors!
rm file
mv file.new file

zypp database

: Either remove it and regenerate it using zypper refresh or just apply; the cp/rm/mv on /var/cache/zypp/zypp.db.

RPM database

: Use the cp/rm/mv trick on /var/lib/rpm/Packages, Basenames, Filemd5s, Dirnames

Sample script

#!/bin/sh
sqlite3 /var/cache/zypp/zypp.db vacuum
rpm --rebuilddb         # takes long
for fn in       /var/cache/zypp/zypp.db \
                /var/lib/rpm/Packages   \
                /var/lib/rpm/Filemd5s   \
                /var/lib/rpm/Dirnames   \
                /var/lib/rpm/Basenames
do
        cp $fn $fn.new || exit 1
        rm $fn
        mv $fn.new $fn
done

zypp,rpm,speed,package management

C C C

openSUSE

Keyboard setting for diacritical characters

In KDE:

Regional settings

keyboard layout

xkb options | enable

compose key position | menu is compose

openSUSE

Monitor an OpenSUSE system

use iostat (package sysstat)
siga (outputs in HTML)
sitar (SLES)

openSUSE

Disable hardware polling for CD-drives

hal cdrom polling hal is a core component of the various desktop environments and deals with all sorts of hardware interaction. One component of the hal daemon is the part where it polls the cdrom drive regularly (as often as once every two seconds!) to see if the user has inserted a CD. This is used, for example, to automatically open a new window with a file browser for the CD.

Such regular polling will keep the hardware awake somewhat; the amount of power consumed depends on the exact type of CDROM drive. It also depends on the presence of the ALPM feature.

If you rarely or never insert CDs (for example because the machine in question is a server located 3000 miles away), you can save some power by stopping this polling.

Current versions of hal have a special command for this:

hal-disable-polling --device /dev/scd0

Note that this means that you will not get a pop-up window if you insert a CD. To enable this polling again, use the this command:

hal-disable-polling --device /dev/scd0 --enable-polling

Newer SATA-based CDROM drives have the capability to notify the machine when a CD gets inserted, making polling unnecessary. Both the kernel and hal are currently undergoing development to detect and support this capability, so that polling is not needed at any time for these devices.

openSUSE

Install all updates using zypper

Install all updates using Zypper

The following command gives you ALL updates:

zypper up -t package

openSUSE

How to locally cache zypper update files

Maybe someday you try zypper dup to actualize your distribution and in middle of process it fail, because you are disconnected or some packages is actualized before you download it (especially on factory this can happen). It is more safety download packages at first and then install from this local files.

How todo this is little tricky, at first you must enable caching downloaded files (I do it only for remote connection):

zypper mr –keep-packages –remote

So now you cache all downloaded files and now try testing run of dup. Trick is that all packages download for that test is cached.

zypper dup –dry-run

Now if you have slow connection I reccomend also disable autorefresh for all repositories, because if repository is refreshed before dup, you can easily find that some packages is newer than package in cache and you must download it.

zypper mr –all –no-refresh

Now is everything prepared for zypper dup, which use files from cache. Cache can take quite lot of disk space, so after dup you can clean it.

zypper clean

And thats all. This features work from OpenSuse 11 and you can also use this trick for zypper update or zypper install.

openSUSE

Check and install media formats

1. Check if there are missing dependencies

Code:

     # LC_ALL=C zypper ve
     Reading installed packages...
     Dependencies of all installed packages are satisfied.

2. Check if there are packages installed from the VideoLAN repository

Code:

$ rpm -qa --queryformat '%{NAME} -> %{VENDOR}\n' | grep -i VideoLAN
libdvdcss -> VideoLAN Project (http://www.videolan.org)

Only libdvdcss should be installed from the VideoLAN repository.

3. Check your basic multimedia packages

Code:

$ rpm --verify --query --all 'libav*' 'libpostproc*' 'libswscale*'

If it return anything there is a problem, check man rpm for details. No need to reinstall to be sure, this check is 100% sure.

4. Verify you have xine installed and installed from Packman

Code:

     $ rpm --query --queryformat '%{NAME}-%{VERSION}-%{RELEASE}-%{ARCH} -> %{VENDOR}\n' libxine1 libxine1-codecs
     libxine1-1.1.15-44.pm.0-x86_64 -> packman.links2linux.de
     libxine1-codecs-1.1.15-44.pm.0-x86_64 -> packman.links2linux.de

5. Check your xine packages

Code:

$ rpm --verify --query --all '*xine*'

6. Verify you have all the gstreamer plugins (codecs) installed and installed from Packman

Code:

     $ rpm --query --queryformat '%{NAME}-%{VERSION}-%{RELEASE}-%{ARCH} -> %{VENDOR}\n' gstreamer-0_10-ffmpeg gstreamer-0_10-fluendo-mp3 gstreamer-0_10-fluendo-mpegdemux gstreamer-0_10-fluendo-mpegmux gstreamer-0_10-plugins-bad gstreamer-0_10-plugins-base gstreamer-0_10-plugins-good gstreamer-0_10-plugins-good-extra gstreamer-0_10-plugins-ugly
     gstreamer-0_10-ffmpeg-0.10.5-42.pm.1-x86_64 -> packman.links2linux.de
     gstreamer-0_10-fluendo-mp3-0.10.7-42.pm.2-x86_64 -> packman.links2linux.de
     gstreamer-0_10-fluendo-mpegdemux-0.10.15-42.pm.1-x86_64 -> http://packman.links2linux.de
     gstreamer-0_10-fluendo-mpegmux-0.10.4-42.pm.1-x86_64 -> packman.links2linux.de
     gstreamer-0_10-plugins-bad-0.10.8-42.pm.1-x86_64 -> packman.links2linux.de
     gstreamer-0_10-plugins-base-0.10.21-42.pm.1-x86_64 -> packman.links2linux.de
     gstreamer-0_10-plugins-good-0.10.10-42.pm.3-x86_64 -> packman.links2linux.de
     gstreamer-0_10-plugins-good-extra-0.10.10-42.pm.3-x86_64 -> packman.links2linux.de
     gstreamer-0_10-plugins-ugly-0.10.10-42.pm.1-x86_64 -> packman.links2linux.de

7. Check your gstreamer packages

Code:

$ rpm --verify --query --all '*gst*'

8. Check your MPlayer package

Code:

$ rpm --verify MPlayer

9. Check your VLC packages

Code:

     $ rpm --verify --query --all '*vlc*'
     .....UG.    /usr/bin/qvlc
     .....UG.    /usr/bin/svlc
     .....UG.    /usr/lib64/vlc/gui/libqt4_plugin.so

Like you see in this case there is a problem, but it's just a permissions problem. Anyway it's a packaging problem I'm going to report to Packman so they can fix it.

10. Check your w32codec package

Code:

$ rpm --verify w32codec-all
package w32codec-all is not installed

Note that this package only adds support for some rare formats, and only works on 32bits systems. On a x86-64 system it will not make any difference.

If after checking all this you are unable to fix your problem open a thread in this forum. But copy & paste (yes, c&p no just "everything seems ok") the commands from this post and your results, so we know you followed them correctly. And also give us a copy of your installed packages... don't say "I have xine from Packman", prove it!!! Code:

$ rpm --query --all 'libav*' 'libpostproc*' 'libswscale*' '*xine*' '*gst*' MPlayer '*vlc*' | grep -v avahi | sort
amarok-xine-1.4.10-100.pm.1
gstreamer-0_10-0.10.21-42.pm.3
gstreamer-0_10-ffmpeg-0.10.5-42.pm.1
gstreamer-0_10-fluendo-mp3-0.10.7-42.pm.2
gstreamer-0_10-fluendo-mpegdemux-0.10.15-42.pm.1
gstreamer-0_10-fluendo-mpegmux-0.10.4-42.pm.1
gstreamer-0_10-lang-0.10.21-42.pm.3
gstreamer-0_10-plugins-bad-0.10.8-42.pm.1
gstreamer-0_10-plugins-bad-lang-0.10.8-42.pm.1
gstreamer-0_10-plugins-base-0.10.21-42.pm.1
gstreamer-0_10-plugins-base-lang-0.10.21-42.pm.1
gstreamer-0_10-plugins-farsight-0.12.9-0.pm.3
gstreamer-0_10-plugins-good-0.10.10-42.pm.3
gstreamer-0_10-plugins-good-extra-0.10.10-42.pm.3
gstreamer-0_10-plugins-ugly-0.10.10-42.pm.1
gstreamer-0_10-plugins-ugly-lang-0.10.10-42.pm.1
gstreamer-0_10-utils-0.10.21-42.pm.3
gstreamer-utils-0.10.21-42.pm.3
kdemultimedia3-video-xine-3.5.10-5.21
libavc1394-0-0.5.3-104.1
libavcodec52-0.4.9.15866svn-20081117.pm.2304
libavdevice52-0.4.9.15866svn-20081117.pm.2304
libavformat52-0.4.9.15866svn-20081117.pm.2304
libavogadro0-0.8.1-18.1
libavutil49-0.4.9.15866svn-20081117.pm.2304
libgstapp-0_10-0-0.10.8-42.pm.1
libgstinterfaces-0_10-0-0.10.21-42.pm.1
libgstreamer-0_10-0-0.10.21-42.pm.3
libpostproc51-0.4.9.15866svn-20081117.pm.2304
libswscale0-0.4.9.15866svn-20081117.pm.2304
libvlc2-0.9.6-0.pm.1
libvlccore0-0.9.6-0.pm.1
libxine1-1.1.15-44.pm.0
libxine1-aa-1.1.15-44.pm.0
libxine1-codecs-1.1.15-44.pm.0
MPlayer-1.0rc2_r27637-3.pm.3
phonon-backend-gstreamer-0_10-4.2.0-53.3
phonon-backend-xine-4.1.3-59.11
vlc-0.9.6-0.pm.1
vlc-noX-0.9.6-0.pm.1
vlc-qt-0.9.6-0.pm.1
xinetd-2.3.14-115.1
xine-ui-0.99.5-161.1

And to avoid problems... the most important thing I can say you: even if YaST/zypper/updater gives you such an option... **NEVER** ignore a dependency. Change vendor is ok, ignore dependencies never is a good idea.

openSUSE

Boot options in suse grub

install=http://provisioning/.......... --> where to boot OS from
autoyast=http://provisioning/......... --> where to get autoyast file from

openSUSE

Zypper tips

To check from which repo package came from

# rpm -qa --qf '%-20{NAME} %-20{VENDOR}\n' 'libav*5[678]'

To install only packages from one repo

#zypper in -r pvdm-kde43 amarok

Change the behaviour of download-install-download-install-download to download all-install all

modify /etc/zypp/zypp.conf and find and change:

commit.downloadMode = DownloadInAdvance

How to locally cache zypper update files

upgrading your system using 'zypper dup'

When you want to do an upgrade, "zypper dup" actually brings your system to the version of the available packages. This may mean downgrading and removing of installed packages.
The recommended way of doing this is to perform it against a specific repo: use

zypper dup -from repo -r, --repo <alias|#|URI> Load only the specified repository.
zypper dup -r 5 -r 6 -r 7

If you intend to use "zypper dup" without any specific repostitory, you need to define repo priorities if you don't want to encounter surprises.

For the full story read http://duncan.mac-vicar.com/blog/archives/619 here.

=if you want to find a certain file:=

zypper in 'libqtiff.so()(64bit)'
Reading installed packages...
'libqt4-x11' providing 'libqtiff.so()(64bit)' is already installed.
Nothing to do.

if you want to install a certain version:

zypper in MozillaFirefox \>= 3

Other examples:

zypper in yast*                     # install all yast modules
zypper in -t pattern lamp_server    # install lamp_server pattern (packages  needed for a LAMP server)
zypper in vim -emacs                # install vim and remove emacs in one go
zypper in amarok packman:libxine1   # install libxine1 from packman and  amarok from any repo
zypper in bitchx-1.1-81.x86_64.rpm  # install bitchx rpm from local directory
zypper in -f subversion             # force reinstallation of subversion

search packages

patterns

list patterns:

zypper patterns

information on packages

zypper if amarok

Information for package amarok:
Repository: @System
Name: amarok
Version: 2.2.0-38.1
Arch: x86_64
Vendor: obs://build.opensuse.org/KDE
Installed: Yes
Status: up-to-date
Installed Size: 24.2 MiB
Summary: Media Player for KDE
Description:
Amarok is a media player for all kinds of media. This includes MP3, Ogg
Vorbis, audio CDs, podcasts and streams. Play lists can be stored in
.m3u or .pls files.

Other examples:

zypper info -t patch MozillaFirefox    # show information about 'MozillaFirefox' patch
zypper patch-info MozillaFirefox       # the same as above
zypper info -t pattern lamp_server     # show info about 'lamp_server' pattern
zypper info -t product openSUSE-FTP    # show info about specified product

dependencies

what-provides or wp

To list all providers of specified capability, do:

 zypper wp firefox
 Reading installed packages...
 S | Name           | Type    | Version     | Arch   | Repository
 --+----------------+---------+-------------+--------+------------------
 i | MozillaFirefox | package | 3.0-0.1     | x86_64 | Updates for 11.1
 v | MozillaFirefox | package | 2.9.95-25.1 | x86_64 | openSUSE-11.1-Oss
 v | MozillaFirefox | package | 3.0-0.1     | i586   | Updates for 11.1
 v | MozillaFirefox | package | 2.9.95-25.1 | i586   | openSUSE-11.1-Oss

Regarding optional dependencies, we do have them, and use them. They're enabled by default. To avoid the several hundred unexpected packages like in the example in the article you can use zypper with --no-recommends or turn them off in the zypper.conf config file.

The zypper summary before an installation distinguishes between hard dependencies and recommends, so you can see what turning recommends off would have saved you even if you didn't.

package locks

To lock all packages starting with 'yast2', do:

zypper al 'yast2*'
Reading installed packages...
Specified lock has been successfully added

To list currently active locks:

To remove a lock, do:

zypper rl yast2-packager

Other examples:

zypper al zypper                   # lock package 'zypper' (exact match)
zypper al -r repo-oss virtualbox*  # restrict the lock to 'repo-oss'  repository (allowing installation from others)
zypper rl 3                        # remove lock by number

rpm examples

# rpm -qi  amarok                 Show details of package
# rpm -qf  /usr/sbin/sshd         Shows where file came from
# rpm -qR openssh-server          Show dependencies - What do I need for this package to work?
# rpm --whatrequires initscripts  Find reverse dependencies - What will break if I remove this package
# rpm -qip                        Show details of not yet installed package
# rpm --verify openssh-server     Find out if any files in the package have been modified since install.
# rpm -qlv postfix                list all files in a rpm

openSUSE

Improve the font rendering - using subpixel hinting

zypper ar -f http://download.opensuse.org/repositories/home:/muzlocker/openSUSE_11.4/ smoothfonts
zypper dup -r smoothfonts

openSUSE

Things to do after a fresh installation of openSUSE 11.4

This guide is a summary of the steps I take to customize a default installation of openSUSE.

This particular guide was rewritten for openSUSE 11.2, but most stuff should be applicable to other versions as well.

first: repos

first, type

# zypper lr -d
# | Alias        | Name                  | Enabled | Refresh | Priority | Type | URI                                                             |  Service
-+--------------+-----------------------+---------+---------+----------+------+-----------------------------------------------------------------+--------
1 | repo-debug   | openSUSE-11.3-Debug   | No      | Yes     |   99     | NONE | http://download.opensuse.org/debug/distribution/11.3/repo/oss/  |
2 | repo-non-oss | openSUSE-11.3-Non-Oss | Yes     | Yes     |   99     | NONE | http://download.opensuse.org/distribution/11.3/repo/non-oss/    |
3 | repo-oss     | openSUSE-11.3-Oss     | Yes     | Yes     |   99     | NONE | http://download.opensuse.org/distribution/11.3/repo/oss/        |
4 | repo-source  | openSUSE-11.3-Source  | No      | Yes     |   99     | NONE | http://download.opensuse.org/source/distribution/11.3/repo/oss/ |
5 | repo-update  | openSUSE-11.3-Update  | Yes     | Yes     |   99     | NONE | http://download.opensuse.org/update/11.3/                       |

delete source and debug repos and installation dvd repo

zypper rr < number of repo | name of repo | alias of repo>

add community repos: Packman, VLC etc.

zypper ar -f http://download.videolan.org/pub/videolan/vlc/SuSE/11.2/ vlc
zypper ar -f http://ftp.skynet.be/pub/packman/suse/11.2/ packman

add KDE 4.3 repos (for updates KDE)

zypper ar -f http://download.opensuse.org/repositories/KDE:/43/openSUSE_11.2/ kde43
zypper ar -f http://download.opensuse.org/repositories/KDE:/KDE4:/Community/openSUSE_11.2_KDE_43/ kde43community

or better still, the updated KDE4.4 repos:

zypper ar -f http://download.opensuse.org/repositories/KDE:/KDE4:/Factory:/Desktop/openSUSE_11.2/ kde44
zypper ar -f http://download.opensuse.org/repositories/KDE:/KDE4:/Community/openSUSE_11.2/ kde44comm
zypper ar -f http://download.opensuse.org/repositories/KDE:/KDE4:/Playground/openSUSE_11.2/ kde44play

In the end you should have the following repos:

oss
non-oss
update
kde43 or kde44
kde43community or kde44comm
packman
vlc
nvidia/ATI

Kernel stuff

Install this stuff now, because sooner or later you will need them. And sometimes the install/compile of a kernel module will fail without these

zypper in -y kernel-source linux-kernel-headers kernel-syms make gcc

utils

zypper in -y mc findutils-locate nmap

For multimedia stuff:

# zypper in -y flash-player mplayer ffmpeg a52dec mencoder x264 faac w32codec-all vlc

optional:

# zypper in -y gstreamer-0_10-plugins-good xine-browser-plugin swfdec-browser-plugin gxine-browser-plugin gstreamer-0_10-plugins-bad mplayerplug-in libxine1-codecs

Language:

zypper in -y ispell-dutch myspell-dutch

If you have installed from liveCD, localization may be missing:

zypper in -y kde4-l10n-nl kde4-l10n-nl-data kde4-l10n-nl-doc

firefox:

install add-ons: adblock, xmarks
add 'new tab' button in buttonbar
configuration:
- download location always ask, close when ready etc
- start with blank page
- do not remember history, suggest from bookmarks
- search for text when I start typing
- no check my spelling as I type
- no check to see if FF is default browser
install silverlight plugin http://go-mono.com/moonlight-beta/
type about:config in addressbar and change:
- network.dns.disableIPv6 -> true
- create this one: content.notify.backoffcount -> 5
- also create nglayout.initialpaint.delay -> 0
- network.http.pipelining true
- network.http.proxy.pipelining true
- network.http.pipelining.maxrequests 8

KDE:

edit /etc/sysconfig/windowmanager and change this option to 'no'

KDE_USE_IPV6="no"

taskbar:
- add system monitor CPU usage widget
- digital clock: add day of week
add yast2 and configure desktop to quick start area (next to suse-icon)
configure desktop - personal settings:
- region and language: region: dutch. add dutch to languages
- advanced -> session manager: start KDE with empty session
- autostart: yakuake
- desktop screensaver: random
desktop: add analogue clock widget
for extra KWIN themes:

# zypper in kde4-windeco-aurorae kde4-windeco-chromi kde4-windeco-crystal kde4-windeco-dekorator kde4-windeco-nitrogen

: and go to System Settings / Appearance / Windows / choose for example the aurorae theme engine with the example theme

Yast2:

configure NTP settings 194.109.22.18 (XS4All time server) or choose from pool.ntp.org

OOo

User details
quickstarter

Printer configuration

cups printer 192.168.1.1 ++ALWAYS USE IP++

Power settings

network mounts

openSUSE

Repos

lap088:/home/pvdm # cat /etc/zypp/repos.d/*
[packman]
name=Packman
enabled=1
autorefresh=1
baseurl=http://packman.inode.at/suse/openSUSE_12.1
type=rpm-md
gpgcheck=1
gpgkey=http://packman.inode.at/suse/openSUSE_12.1/repodata/repomd.xml.key
keeppackages=0
[pvdm-garmin]
name=pvdm-garmin
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/repositories/Application:/Geo/openSUSE_12.1/
type=rpm-md
[pvdm-kde48-extra]
name=pvdm-kde48-extra
enabled=1
autorefresh=1
baseurl=http://pvdm.xs4all.nl/kde48extra/
type=rpm-md
keeppackages=0
[pvdm-kde48]
name=pvdm-kde48
enabled=1
autorefresh=1
baseurl=http://pvdm.xs4all.nl/kde48/
type=rpm-md
keeppackages=0
[pvdm-kernel]
name=pvdm-kernel
enabled=1
autorefresh=1
baseurl=http://pvdm.xs4all.nl/kernel/
type=rpm-md
keeppackages=0
[pvdm-nntpgrab]
name=pvdm-nntpgrab
enabled=0
autorefresh=1
baseurl=http://www.openftd.org/releases/openSUSE_12.1
type=rpm-md
keeppackages=0
[pvdm-smoothfonts]
name=pvdm-smoothfonts
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/repositories/home:/muzlocker/openSUSE_12.1/
type=rpm-md
keeppackages=0
[repo-non-oss]
name=Non-Oss
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/distribution/12.1/repo/non-oss/
path=/
type=yast2
keeppackages=0
[repo-oss]
name=Oss
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/distribution/12.1/repo/oss/
path=/
type=yast2
keeppackages=0
[Updates]
name=PvdM Updates
enabled=1
autorefresh=1
baseurl=http://pvdm.xs4all.nl/update/12.1/
path=/
type=rpm-md
keeppackages=0

openSUSE

Touchpad

hp.com tydelijk1 uitgebruide ww met CAPS johan(2) en cijfers

borderlinx pvdmjoomla uit cijf

openSUSE

Things to do after a fresh installation of openSUSE 12.1

This guide is a summary of the steps I take to customize a default installation of openSUSE.

This particular guide was rewritten for openSUSE 11.2, but most stuff should be applicable to other versions as well.

first: repos

first, type

# zypper lr -d

delete source and debug repos and installation dvd repo

zypper rr < number of repo or name of repo or alias of repo>

add community repo: Packman

zypper ar -f http://packman.inode.at/suse/openSUSE_12.1 packman

In the end you should have the following repos:

# zypper lr --uri
# | Alias                              | Name                               | Enabled | Refresh | URI                                                         
--+------------------------------------+------------------------------------+---------+---------+-------------------------------------------------------------
1 | Updates-for-openSUSE-12.1-12.1-1.4 | Updates for openSUSE 12.1 12.1-1.4 | Yes     | Yes     | http://download.opensuse.org/update/12.1/                   
2 | packman                            | Packman repository (openSUSE_12.1) | Yes     | No      | http://packman.inode.at/suse/openSUSE_12.1                  
3 | repo-non-oss                       | openSUSE-12.1-Non-Oss              | Yes     | Yes     | http://download.opensuse.org/distribution/12.1/repo/non-oss/
4 | repo-oss                           | openSUSE-12.1-Oss                  | Yes     | Yes     | http://download.opensuse.org/distribution/12.1/repo/oss/

Kernel stuff

Install this stuff now, because sooner or later you will need them. And sometimes the install/compile of a kernel module will fail without these

zypper in -y kernel-source linux-kernel-headers kernel-syms make gcc

utils

zypper in -y mc findutils-locate nmap yakuake krusader chromium clementine vlc unetbootin p7zip krename

For multimedia stuff & codecs:

# zypper in -y flash-player flashplayer-kde4 mplayer ffmpeg a52dec mencoder x264 faac w32codec-all lame kdemultimedia4-thumbnailers

optional:

# zypper in -y gstreamer-0_10-plugins-good xine-browser-plugin swfdec-browser-plugin gxine-browser-plugin gstreamer-0_10-plugins-bad gstreamer-0_10-plugins-ugly mplayerplug-in libxine1-codecs k3b-codecs gstreamer-0_10-ffmpeg

for windows movies in webbrowser:

# zypper in gnome-mplayer gecko-mediaplayer

fonts

To install MS fonts so that LibreOffice documents looks closer to those on my colleagues' screen, execute the following (as root):

zypper in fetchmsttfonts

Language:

zypper in -y ispell-dutch myspell-dutch

If you have installed from liveCD, localization may be missing:

zypper in -y kde4-l10n-nl kde4-l10n-nl-data kde4-l10n-nl-doc

firefox:

install add-ons: adblock, xmarks
add 'new tab' button in buttonbar
configuration:
- download location always ask, close when ready etc
- start with blank page
- do not remember history, suggest from bookmarks
- search for text when I start typing
- no check my spelling as I type
- no check to see if FF is default browser
==>install silverlight plugin http://go-mono.com/moonlight-beta/<== maybe not
type about:config in addressbar and change:
- network.dns.disableIPv6 -> true
- create this one: content.notify.backoffcount -> 5
- also create nglayout.initialpaint.delay -> 0
- network.http.pipelining true
- network.http.proxy.pipelining true
- network.http.pipelining.maxrequests 8

KDE:

edit /etc/sysconfig/windowmanager and change this option to 'no'

KDE_USE_IPV6="no"

(probably already is)

taskbar:
- add system monitor CPU usage widget
- digital clock: add day of week
add yast2 and configure desktop to quick start area (next to suse-icon)
configure desktop - personal settings:
- region and language: region: dutch. add dutch to languages
- advanced -> session manager: start KDE with empty session
- autostart: yakuake konversation pidgin
- desktop screensaver: random
desktop: add analogue clock widget
configure powermanagement
configure kwin
configure fonts

Yast2:

configure NTP settings 194.109.22.18 (XS4All time server) or choose from pool.ntp.org

OOo

User details
quickstarter

Printer configuration

cups printer 192.168.1.1 ++ALWAYS USE IP++

Power settings

network mounts

openSUSE

Radeon

- The Radeon hardware was tested with the following non-stock options:

disabling swap buffers wait from the xorg.conf (SwapbuffersWait),
enabling 2D color tiling (ColorTiling and ColorTiling2D in xorg.conf),
and enabling PCI Express 2.0 support (radeon.pcie_gen2=1 as a kernel command-line parameter).

It is silly that these non-default options are still needed for the open-source driver that AMD officially supports, but that is how it stands right now.

- With the Nouveau driver the main performance-boosting item at this point is forcing the graphics card manually into its highest performance level / clock state. The Nouveau driver continues to leave the graphics core / memory / shader clocks at whatever the video BIOS to sets them at when initializing, unlike the NVIDIA binary driver that dynamically changes the performance levels based upon GPU load and other factors. Therefore, with most modern NVIDIA hardware having multiple performance levels, to get the GPU at the correct frequencies for 3D use you need to manually force them to the higher state. Meanwhile you need to hope that the NVIDIA driver properly supports re-clocking for your GPU and that once re-clocked you don't experience any rendering corruption or stability problems. Re-clocking involves loading the Nouveau DRM driver with a special command-line parameter and writing the desired performance level to a sysfs node. This information is covered in detail in Nouveau Reclocking: Buggy, But Can Boost Performance. I long for the day when the Nouveau driver properly supports re-clocking across all modern NVIDIA GPUs and can be enabled by default to be done automatically.

openSUSE

How to have the latest kernel in openSUSE

It is possible to have the latest kernel in openSUSE. However, there are a few thing you must consider. These will be discussed below.

We will be using the command line tool zypper to update to the latest kernel, because it is easier than YaST :D

stock kernels

OpenSUSE is installed (from DVD or wherever from) with a kernel called the 'stock' kernel. Sometimes this kernel is also referred to the GA kernel. This kernel remains the same for the whole time that that specific openSUSE version is supported, and is only updated with security issues and occasional bug fixes. This is called backporting. These updated stock-kernels appear in the update channel just as normal updates, so you should install them, just as other updates.

This is a list of the latest openSUSE versions and their stock kernels:

openSUSE version:    stock-kernel
12.2                 3.4.6-2.10
12.1                 3.1.9 upgraded to 3.1.10 for security/bugfixes
11.4                 2.6.37

kernel versions

openSUSE has the option to run other 'flavours' of the kernel. These include:

kernel-default -> The standard kernel for both uniprocessor and multiprocessor systems.
kernel-desktop -> This kernel is optimized for the desktop. It is configured for lower latency and has many of the features that aren't usually used on desktop machines disabled.
kernel-pae -> this is a 32-bit only special kernel with support for 64GB memory. 64 bit kernels support this by default
kernel-vanilla -> The standard kernel - without any SUSE patches
kernel-xen -> The Linux kernel for Xen paravirtualization. This kernel can be used both as the domain0 ("xen0") and as an unprivileged ("xenU") kernel.

Usually, for a desktop/laptop system, you run the default or the desktop flavour.

keeping older kernel versions as fallback

It is good practice, when you enable the kernel-head repo and thus recieve the latest and greatest kernel, to keep some previous kernel(s) at hand to use as a fallback scenario. These kernels will appear in your grub menu so you will be able to choose them when booting up your machine.

Fortunately, zypper supports this with a few configuration options. It is possible for instance to instruct zypper to keep the stock kernel, the latest kernel, and the latest but one and to remove all others. That is cool, right?

The magic is in the file /etc/zypp/zypp.conf. There are two options there:

multiversion = provides:multiversion(kernel)
multiversion.kernels = latest,oldest

The first line enables the mutiple installation of kernels simultaniously. This is not possible by default, so this option is necessary.

The second has several options:

## Comma separated list of kernel packages to keep installed in parallel, if the
## above multiversion variable is set. Packages can be specified as
## 2.6.32.12-0.7 - Exact version to keep
## latest        - Keep kernel with the highest version number
## latest-N      - Keep kernel with the Nth highest version number
## running       - Keep the running kernel
## oldest        - Keep kernel with the lowest version number (the GA kernel)
## oldest+N      - Keep kernel with the Nth lowest version number
##
## Default: Do not delete any kernels if multiversion = provides:multiversion(kernel) is set
#multiversion.kernels = latest,oldest

So, if you want to keep the stock kernel and the latest kernel, use:

multiversion.kernels = latest,oldest

If you want to keep the stock kernel, the latest and the kernel before that, use:

multiversion.kernels = latest,oldest,latest-1

enabling the kernel head repository

To receive the latest kernel, you must create an additional repo to download from.

zypper ar -f http://download.opensuse.org/repositories/Kernel:/HEAD/standard/ kernel-repo

This command add a new repository (ar), enables refresh (-f), provides the URL, and a name (kernel-repo).

This repository always contains the lastest development kernel, i.e. 3.4.0-rc6-1-desktop. Do not expect to find older versions there, it is always the latest and only the latest build!

This repo is the same for all openSUSE version,

updating from the new repo

To actually download and install the new kernel, you must do a special update. This is called a distribution-upgrade. Don't worry, this won't update your entire installation! We will instruct zypper to only use updated packages from the kernel-repo we just created. A normal 'zypper up' won't replace packages from a different repo then the one it was originally installed from.

Issue the command:

zypper dup -r kernel-repo

This tells zypper to do a dist-upgrade from the kernel-repo, so only your new kernel files will be installed alongside the old ones, if you have properly configured the zypp.conf file.

check check double check

After you have completed these instructions, you can check which kernels you have installed:

rpm -qa | grep kernel

The output of that command on my system at time of this writing:

#rpm -qa | grep kernel | sort
kernel-default-devel-3.4.rc3-3.1.x86_64
kernel-default-devel-3.4.rc6-1.1.x86_64
kernel-desktop-3.4.rc3-3.1.x86_64
kernel-desktop-3.4.rc6-1.1.x86_64
kernel-desktop-devel-3.4.rc3-3.1.x86_64
kernel-desktop-devel-3.4.rc6-1.1.x86_64
kernel-devel-3.4.rc3-3.1.noarch
kernel-devel-3.4.rc6-1.1.noarch
kernel-source-3.4.rc3-3.1.noarch
kernel-source-3.4.rc6-1.1.noarch
kernel-syms-3.4.rc3-3.1.x86_64
kernel-syms-3.4.rc6-1.1.x86_64
kernel-xen-devel-3.4.rc3-3.1.x86_64
kernel-xen-devel-3.4.rc6-1.1.x86_64

As you can see, I have 2 kernels installed at the moment: 3.4.rc3 and 3.4.rc6. (The rest of the packages are necessary for other purposes, like Virtualbox.)

openSUSE

Fixing desktop freezes on large copies

Now this may or may not be specific to openSUSE, but here it is anyway:

I suffered from desktop lockups when copying large files, for instance from the wireless network to usb. THis was especially noticable in dolphin.

Even though I suspected it at first, now it appears that it is not dolphin that is acting up, it is the 'hugepage' option in the kernel.

So, how do we tweak it? Simpel. The following lines will do the trick as root:

echo never >/sys/kernel/mm/transparent_hugepage/defrag
echo 0 > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag

-edit-

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled

openSUSE

KDE font settings

Fonts:

3,4 & 6

openSUSE

How to make the screen brightness keys work again in KDE

How to make button for screen brightness work on your laptop in KDE

When I started using a newer kernel in openSUSE 13.2, I noticed the buttons for screen brightness were no longer functioning on my laptop. This is verry annoying, so I decided to do something about it. I am using KDE. I will try to write the instructions as generic as possible, so that it will hopefully work in your situation too. This is a HP ELitebook 8570p.

find the right entry

First, we have to find the screen device, which is located somewhere in the /sys/devices/pci0000:00 directory.

Somewhere in this directory is a file located with the name 'brightness'. Let's find it. So, as root:

# find /sys/devices/pci0000:00 -name "brightness"
/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness
/sys/devices/pci0000:00/0000:00:1c.2/0000:24:00.1/leds/mmc0::/brightness
/sys/devices/pci0000:00/0000:00:1c.3/0000:25:00.0/leds/phy0-led/brightness

Remember, your output is probably different from what I got here.

I have a radeon card, so the first line contains the device we will be working with.

When you read the content of the 'file', you get the current brightness level:

# cat /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness
235

The value lies somewhere between 0 and 255, it's a one byte value.

You can test if this work for you by writing (still as root) a different value to that file: (remember to substitute the file for your graphics card)

# echo 200 > /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness

If this works, you should see the brightness change, and you can go to the next step.

edit: for samsung rv720: "echo 8 > /sys/class/backlight/samsung/brightness" to restore the brightness. edit2: newer kernel uses this: echo 100 > /sys/class/backlight/nv_backlight/brightness

create the scripts

I created a simple QAD(quick and Dirty) 3 line script to read that value and increase it.

I put the script in my home user's bin directory:

user> vi ~/bin/brp

br=$(cat /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness)
br=$((br+10))
echo $br > /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness

I called it 'brp' which is nice and short and stands for 'brightness plus' Note that I increase the value by 10 otherwise it would not change enough. You can vary that value of course to get finer control or get faster adjustment.

I created a similar script called brm which decreases the value:

user> vi ~/bin/brm

And paste the content

br=$(cat /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness)
br=$((br-10))
echo $br > /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness

# chmod +x brm brp

Now, the problem is that this file /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness may only be written by root. Notice that all commands manipulating the file so far have been executed as root. But you should have created the 2 script files brp and brm as your normal user account.

To make the file world writeable:

# chmod 777 /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness

This will make the 2 scripts which are executed as your useraccount able to write to this file.

configure KDE

Then, lastly, redefine the keys on your laptop for screen brightness to point to your script:

go to KDE - configure desktop
shortcuts and gestures
custom shortcuts
edit - new - global shortcut - command/URL
name: br-down
tab trigger: Shortcut - click on 'None'
press the brightness down key on your keyboard. It should appear in the button.
tab action: browse and point to your brm file (/home/$user/bin/brm)
click apply

Repeat this for the brightness up script file, so you end up with 2 shortcuts.

And that's it. This should work. Happy screening!

configure crontab

Oh, and after a reboot, the chmod 777 of the brightness file is needed again. So I did a dirty hack and added this to my crontab:

*/3 * * * *     root    chmod 777 /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card0/card0-LVDS-1/radeon_bl0/brightness

openSUSE

Snapper

Introduction

This article assumes you will be using Snapper with the Btrfs filesystem.

During a fresh install of openSUSE, the default is to use Btrfs on root, meaning / and the filesystem XFS on /home. If you select the option to not seperate /home during openSUSE install, the default is for everything, including /home, to be under Btrfs.

If you decide to use Btrfs & Snapper (default) during a fresh openSUSE install, it will automatically configure Snapper for you. This automatic configuration of Snapper means that snapshots will be made automatically when you use YaST2 and/or zypper. Please read section Software Requirements below for caveat.

You may convert EXT3 & EXT4 filesystems to Btrfs to take advantage of Snapper. Using Snapper on EXT4 filesystem is highly discouraged.

Important Details

The default behaviour when Snapper is configured to run on root, meaning /, is to exclude every Btrfs subvolume. This the same behaviour as on SUSE Enterprise. This means that if you have the subvolume /srv Snapper will not snapshot anything in /srv unless told specifically by you to do so. This also means that if you add a subvolume to / at any point in time, it will be excluded from Snapper's default snapshot behaviour.

Always be root when using Snapper or YaST2, unless otherwise specified.

YaST2 comes with a Snapper module called yast2-snapper and may be used to check and manipulate snapshots. It does not, however, have the ability to change Snapper configurations. That must be done in the terminal.

Taking snapshots is automatically enabled if the root partition (/) is big enough (approximately more than 8GB). Please remember: taking snapshots on partitions other than / is not enabled by default.

To see what subvolumes are created under / and therefore see which directories (subvolumes) are excluded from the default snapshots behaviour:

# btrfs subvolume list /

To have Snapper create a snapshot of a subvolume, you can either do it manually one-time via the snapper program, or create a new snapper config file. More information in the tutorial.

Software Requirements

You always need this package:

snapper

If you use YaST2 to install, update or remove packages and want to have snapper automatically create snapshots when you use this tool:

yast2-snapper

If you use zypper to install, update or remove packages and want to have snapper automatically create snapshots when you use this tool:

snapper-zypp-plugin

If you want to have the ability for advanced btfs snapshot boot menu management:

grub2-snapper-plugin

WARNING: the above packages are not always installed by default. What gets installed depends on what choices you make during the openSUSE installation process. Make sure you have the appropriate (usually that means all) packages listed above installed before making changes to your system, otherwise you may be in for a surprise when snapper does not automatically create snapshots you thought it would.

Snapper Basic Usage

Snapper is not restricted to creating and managing snapshots automatically by configuration; you can also create snapshot pairs ("before and after") or single snapshots manually using either the command line tool or the YaST module.

All Snapper operations are carried out for an existing configuration. You can only take snapshots of partitions or volumes for which a configuration exists. By default the system configuration (root) is used. If you want to create or manage snapshots for your own configuration you need to explicitly choose it. Use the Current Configuration drop-down box in YaST or specify the -c on the command line (snapper -c MYCONFIG COMMAND).

List all snapshots for the default configuration (root)

snapper list

snapper ls

Show which files and directories have been changed between snapshots

For files/directories changed between snapshot 21 and 22:

snapper status 21..22

You can also show what files and directories have been changed from snapshot 41 going back to 39:

snapper status 41..39

The output consists of a string encoding the status followed by the filename. The characters of the status string are:

"+" means the file was created
"-" means the file was deleted
"c" means the content of the file has changed
"t" means the type of the file has changed (e.g. from regular file to directory)
"p" means the permissions are have changed
"u" means the user ownership has changed
"g" means the group ownership has changed
"x" means the extended attribute information has changed
"a" means the ACL information has changed
"." means no change

Show the diff (difference in actual files) between snapshots

For diff between snapshot 71 and 72:

snapper diff 71..72

You can also use this command to show the difference between a specific file. For example to show the diff from snapshot 71 to 72 only for file /etc/zypp/zypp.conf you would do the following:

snapper diff 71..72 /etc/zypp/zypp.conf

Cleanup Algorithms

Unless you have a good reason to do otherwise, you should always specify the cleanup algorithm when creating a snapshot, otherwise the snapshot will never be deleted unless you do it manually. You do this by adding the following to your snapper commands:

--cleanup-algorithm <number|timeline|empty-pre-post>

The following commands assume you will be creating snapshots on the default system configuration (root). As mentioned above, if you want to use the non-default configuration, add the following the the snapshot commands:

-c MYCONFIG

Create a snapshot of the type pre and prints the snapshot number. First command needed to create a pair of snapshots used to save a "before" and "after" state.

snapper create --type pre --print-number --description "Before the Apache config cleanup" --cleanup-algorithm number

Create a snapshot of the type post paired with the pre snapshot number 30. Second command needed to create a pair of snapshots used to save a "before" and "after" state.

snapper create --type post--pre-number 30 --description "After the Apache config cleanup" --cleanup-algorithm number

Creates a stand-alone snapshot (type single) for the default (root) configuration with a description. Because no cleanup-algorithm is specified, the snapshot will never be deleted automatically.

snapper create --description "Snapshot for week 2 2014"

Deleting Snapshots

Delete snapshot 65 for the default (root) configuration:

snapper delete 65

Snapper Tutorial

First let us check how YaST has configured snapper:

# snapper list-configs
Config | Subvolume
-------+----------
root | /

As you can see YaST has created a snapper config called "root" for your root file system. You can see what snapshots exist:

Snapshot #0 always refers to the current system. There might already be several other snapshots depending on the uptime of your system and on whether you already used YaST or zypper.

Now we want to try snapper as a undo tool for YaST. First we run YaST:

# yast2 system_settings

Within YaST enable the SysReq. After you have finished YaST you see two new snapshots:

Types of Snapshots

It's time to explain the type of snapshots. Snapper creates a snapshot before and after YaST runs, these snapshots are called pre and post respectively. The post snapshots knows which pre snapshots belongs to it. By having a pre and post snapshot we can see what changes happened to the file system while YaST was running. Single snapshots have no special relationship to other snapshots.

Seeing What Has Changed (YaST)

Using snapper, it is easy to see what has changed while YaST was running. To do so you have to pass the number of the pre and post snapshot:

# snapper status 2..3
c... /etc/sysctl.conf

The 'c' means that the content of the file has changed.

We can also see the diff of the file:

# snapper diff 2..3
--- /.snapshots/2/snapshot/etc/sysctl.conf 2011-11-22 10:35:43.355493753 +0100
+++ /.snapshots/3/snapshot/etc/sysctl.conf 2011-11-22 10:41:47.019512185 +0100
@@ -17,7 +17,7 @@
# See sysctl.conf(5) and sysctl(8) for more information
#
####
-kernel.sysrq = 0
+kernel.sysrq = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.forwarding = 0

So, if you don't like the change done by YaST and want to revert it call:

# snapper undochange 2..3
create:0 modify:1 delete:0
undoing change...
undoing change done

But note that snapper does not tell the kernel about the change like YaST did so you must either do so yourself or reboot.

Adding /home To Snapper

During installation YaST does not setup a snapper config for /home. We can do so manually:

# snapper -c home create-config /home

Whenever you want to use snapper for /home you must provide the option -c home.

Now you can see what files have changed since the last hourly snapshot:

# snapper -c home status 1..0
comparing snapshots... done
+... /home/tux/just-married.jpg

The '+' sign means that the file is new.

Manually Creating A Snapshot

If you manually want to create a snapshot use:

# snapper -c home create --description "before the big cleanup"

YaST Snapper GUI

Finally yast2-snapper provides a YaST UI for snapper.

Troubleshooting

Logs

Snapper logs can be found in /var/log/snapper.log, check this file if you are having issues as it may contain useful troubleshooting information.

Snapper Videos

Snapper Quick Overview (4min)

Greg Kroah-Hartman & SUSE Linux Enterprise Product Manager Demo Snapper (~7min)

Snapper, Btrfs & LVM Presentation (22min)

openSUSE

Ansible

openSUSE

Snapper PvdM

Show configs:

# snapper list-configs

Create snapshot:

# snapper -c home create --description "/home snapshot"

View snapshots:

# snapper -c home ls

delete snapper config

# snapper -c home delete-config

If that does not work:

remove /home from fstab
reboot and log in as root

cd {vol}/.snapshots (b.v. /home/.snapshots)

# for i in $(ls -1); do btrfs property set -ts $i/snapshot/ ro false; done

```
# rm -rf *
```
```
# ..
```
```
# rmdir .snapshots
```

openSUSE

Kde over ssh

$ export XDG_CURRENT_DESKTOP=kde
$ dolphin

openSUSE

Change root's theme

$ pkexec env "DISPLAY=$DISPLAY" "XAUTHORITY=$XAUTHORITY" "QT_QPA_PLATFORMTHEME=kde" systemsettings

then change theme as normal.

openSUSE

Upgrade using patch

Using patch

A small patch file is usually made available for a minor version upgrade. You'll need to download patch to use this Manually download and extract the patch file from the dumps site or follow the directions with wget below. Patches are incremental, you can not skip a version.

cd to your main MediaWiki directory (the one with LocalSettings.php)

Download the patch file and unzip it.

Use patch -p1 --dry-run to check what will be changed (e.g., patch -p1 --dry-run -i mediawiki-x.xx.x.patch)

If all is well, run patch again without --dry-run

Check your Special:Version and you should see the new version number in place.

openSUSE

KDE apps look ugly and missing icons over ssh

 Re: Qt5 apps look ugly over SSH X forwarding
Sat May 06, 2017 8:10 am
In the remote SSH session:

Code: Select all
    export XDG_CURRENT_DESKTOP=kde

openSUSE

Things to do after a fresh installation of OpenSUSE

This guide is a summary of the steps I take to customize a default installation of openSUSE.

This particular guide was rewritten for openSUSE 11.2, but most stuff should be applicable to other versions as well.

first: repos

first, type

delete source and debug repos and installation dvd repo

zypper rr < number of repo | name of repo | alias of repo>

add community repos: Packman, VLC etc.

zypper ar -f http://download.videolan.org/pub/videolan/vlc/SuSE/11.2/ vlc
zypper ar -f http://ftp.skynet.be/pub/packman/suse/11.2/ packman

add KDE 4.3 repos (for updates KDE)

or better still, the updated KDE4.4 repos:

In the end you should have the following repos:

oss
non-oss
update
kde43 or kde44
kde43community or kde44comm
packman
vlc
nvidia/ATI

Kernel stuff

Install this stuff now, because sooner or later you will need them. And sometimes the install/compile of a kernel module will fail without these

zypper in -y kernel-source linux-kernel-headers kernel-syms make gcc

utils

zypper in -y mc findutils-locate nmap

For multimedia stuff:

# zypper in -y flash-player mplayer ffmpeg a52dec mencoder x264 faac w32codec-all vlc

optional:

# zypper in -y gstreamer-0_10-plugins-good xine-browser-plugin swfdec-browser-plugin gxine-browser-plugin gstreamer-0_10-plugins-bad mplayerplug-in libxine1-codecs

Language:

zypper in -y ispell-dutch myspell-dutch

If you have installed from liveCD, localization may be missing:

zypper in -y kde4-l10n-nl kde4-l10n-nl-data kde4-l10n-nl-doc

firefox:

install add-ons: adblock, xmarks
add 'new tab' button in buttonbar
configuration:
- download location always ask, close when ready etc
- start with blank page
- do not remember history, suggest from bookmarks
- search for text when I start typing
- no check my spelling as I type
- no check to see if FF is default browser
install silverlight plugin http://go-mono.com/moonlight-beta/
type about:config in addressbar and change:
- network.dns.disableIPv6 -> true
- create this one: content.notify.backoffcount -> 5
- also create nglayout.initialpaint.delay -> 0
- network.http.pipelining true
- network.http.proxy.pipelining true
- network.http.pipelining.maxrequests 8

KDE:

edit /etc/sysconfig/windowmanager and change this option to 'no'

KDE_USE_IPV6="no"

taskbar:
- add system monitor CPU usage widget
- digital clock: add day of week
add yast2 and configure desktop to quick start area (next to suse-icon)
configure desktop - personal settings:
- region and language: region: dutch. add dutch to languages
- advanced -> session manager: start KDE with empty session
- autostart: yakuake
- desktop screensaver: random
desktop: add analogue clock widget
for extra KWIN themes:

# zypper in kde4-windeco-aurorae kde4-windeco-chromi kde4-windeco-crystal kde4-windeco-dekorator kde4-windeco-nitrogen

: and go to System Settings / Appearance / Windows / choose for example the aurorae theme engine with the example theme

Yast2:

configure NTP settings 194.109.22.18 (XS4All time server) or choose from pool.ntp.org

OOo

User details
quickstarter

Printer configuration

cups printer 192.168.1.1 ++ALWAYS USE IP++

Power settings

network mounts

openSUSE

Updating OpenSUSE in-place using zypper dup

What Why When How

If you want to upgrade your OpenSUSE while the system is running, you can use 'zypper dup' with these simple 5 steps.
After the reboot, the system will be upgraded to the next version.
We will be using the command line tool, because it is cool.
This guide can be used with all versions from 11.1 and newer.
The examples given here are from the upgrade of 11.1 to 11.2
A drawback of this method is that your disk will not be converted to use the new ext4 filesystem. For this, do a fresh/clean install

1. Update the system first

Update your current running system with the latest patches. This will ensure that the update process will function correctly and no bugs exist in the rpm/zypper stack. If your system is already patched, continue with the next step If you haven't done so, enable an update repository and bring your system up-2-date.

# zypper addrepo --check --name 'openSUSE-11.1-Update' http://download.opensuse.org/update/11.1/ repo-update
# zypper ref -f && zypper up

After the update a reboot may be required. Make sure this step is successful before proceeding!

2. Delete non standard repos

In this step we need to disable all non-standard OpenSUSE repositories, leaving only the OSS and NON-OSS and an update channel.

# zypper lr -d

Here you will see the repos and their URLs. remove *ALL* the URLS that are not the stanard repos, so remove PackMan, VLC, buildservice repos, google repos, KDE repos, etc etc....

# zypper rr

replace the number with the number displayed in the first column of the 'zypper lr -d' command In the end you will have just 3 repos: update, OSS and NONOSS.

3. Edit the repos with the new location

Now we will bring the 3 repo's URLs to the desired new version.

# cd /etc/zypp/repos.d/

use vi or your favourite editor to edit the 3 files there.

Modify the line with 'baseurl' to use the new location, so change the 11.1 into 11.2. While you're there, you can also modify the line with 'name' and maybe the first line between the brackets.

When you're done, you should have 3 files representing the 3 new repos.

4. Update the zypper program itself first

Enter the following commands:

# zypper ref -f
# zypper in zypper

5. Do the actual upgrade

We will download all packages locally first, to make sure the upgrade goes smoothly and uninterrupted.

Make sure you have enough space in /var !! Check with

# df -h

and see if the free space on / (or /var) is more then the next command states at the bottom summary, before you give the final go:

# zypper dup -d

Then start the upgrade:

# zypper dup

After this, you can add the repos you had before, like Packman etc. Make sure you add the repos for the new version. Then, do another 'zypper dup', use dup because packages may require a vendor change.

openSUSE

Repositories in openSUSE - The current state

#  | Alias     | Name      | Enabled | Refresh | Priority | Type   | URI                                                                               
---+-----------+-----------+---------+---------+----------+--------+-----------------------------------------------------------------------------------
1  | OSS       | OSS       | Yes     | Yes     |   99     | yast2  | http://download.opensuse.org/distribution/11.2/repo/oss/                          
2  | VLC       | VLC       | Yes     | Yes     |   99     | rpm-md | http://download.videolan.org/pub/videolan/vlc/SuSE/11.2/                                  
3  | Wine      | Wine      | Yes     | Yes     |   99     | rpm-md | http://download.opensuse.org/repositories/Emulators:/Wine/openSUSE_11.2/                  
4  | ati       | ati       | Yes     | Yes     |   99     | rpm-md | http://www2.ati.com/suse/11.2/                                                            
5  | faccont   | FacCont   | Yes     | Yes     |   99     | rpm-md | http://download.opensuse.org/repositories/openSUSE:Factory:Contrib/standard               
6  | moz       | moz       | Yes     | Yes     |   99     | rpm-md | http://download.opensuse.org/repositories/mozilla/openSUSE_11.2/                          
7  | nonoss    | NonOSS    | Yes     | Yes     |   99     | yast2  | http://download.opensuse.org/distribution/11.2/repo/non-oss/                              
8  | ooounst   | ooounst   | Yes     | Yes     |   99     | rpm-md | http://download.opensuse.org/repositories/OpenOffice.org:/UNSTABLE/openSUSE_11.2/         
9  | openftd   | openftd   | Yes     | Yes     |   99     | rpm-md | http://www.openftd.org/releases/testing/openSUSE_11.2                                     
10 | p-kde44   | p-kde44   | Yes     | Yes     |   98     | rpm-md | http://pvdm.xs4all.nl/kde44/                                                              
11 | p-kdecomm | p-kdecomm | Yes     | Yes     |   99     | rpm-md | http://pvdm.xs4all.nl/kde44community                                                      
12 | p-kdeplay | p-kdeplay | Yes     | Yes     |   99     | rpm-md | http://pvdm.xs4all.nl/kde44playground                                                     
13 | p-oss     | p-oss     | Yes     | Yes     |   99     | yast2  | http://pvdm.xs4all.nl/opensuse112_64/                                                     
14 | p-up      | p-up      | Yes     | Yes     |   99     | rpm-md | http://pvdm.xs4all.nl/update/11.2/                                                        
15 | packman   | Packman   | Yes     | Yes     |   99     | rpm-md | http://ftp.skynet.be/pub/packman/suse/11.2/

openSUSE

Software regelmatig te checken

ampache
gallery
mediawiki

openSUSE

Upgrading the system to a newer version using zypper dup

Warning: this procedure only works on openSUSE 11.1+

This procedure I have written is based on the official openSUSE wiki. I have made it more compact and readable.

As root:

1. check that you have an update repo, that is is enabled, and update the system using

#zypper up

2. disable all current repos using

#zypper mr --all --disable

3. add the new repos, for 11.3 the commands are:

zypper addrepo --name "openSUSE-11.3 OSS" http://download.opensuse.org/distribution/11.3/repo/oss/ repo-11.3-oss
zypper addrepo --name "openSUSE-11.3 Non-OSS" http://download.opensuse.org/distribution/11.3/repo/non-oss/ repo-11.3-non-oss
zypper addrepo --name "openSUSE-11.3 Updates" http://download.opensuse.org/update/11.3/ repo-11.3-update

4. add 3rd party repos you had before, like packman, but be careful not to add conflicting repos. Check they are the 11.3 version!

5. perform a repo refresh using:

# zypper ref -f

6. do the actual upgrade, by downloading all the packages first and then installing (this is safer):

# zypper dup --download "in-advance"

: Be sure to check the 'packages to downgrade' and 'packages to remove' part of the list.

6a. I noticed, and this is not in the official wiki, that the last commandline did not do the actual install, it did only the download. So you have to run

# zypper dup

7. afterwards you can review all the repos (using # yast2 repositories) and remove the ones from the old version we disabled in step 2.

8. add any repo you miss here and you're done!

Homelab

Boot & Disk

Reinstalling GRUB after Windows install

Enable harddisk spin-down / sleep mode / standby

Linux Software RAID: growing filesystems and adding disks

Adding partitions

Expanding existing partitions

Extending an existing RAID array

Extending the filesystem

LVM: Growing the PV

Partitions

Using GRUB to boot multiple operating systems, the correct way

chainloader

MBR info

What You Absolutely Need to Know

Booting Up Foreign Distros

The Single Partition Configuration

Having Grub Do Your Research For You

Making a Full grub Boot Floppy

Installing grub From Floppy

Boot from the grub floppy

Copy files from the floppy to /boot/grub on the hard disk

Configure /boot/grub/menu.lst for this computer

Reboot from floppy, and install grub

Summary

Working With a Fully Installed grub

Options from the grub menu

Why Dedicated?

Creating Your Grub Partition

Setting Up Your Grub Partition

Making Your menu.lst

Making it Happen at Boot Time

Adding a New Distro

The Case for Doing this On Every Hard Disk

Install Linux on an SSD

1. start your partitons on a 4k boundary

2. Optimize I/O

Use a different scheduler

3. Use TRIM

Re-using an SSD

Further performance enhancements

Btrfs

Fix grub

grub edit

Usb drives

GRUB

How to Rescue a Non-booting GRUB 2 on Linux [1]

GRUB 2 Command Shell

Booting From grub>

Booting From grub-rescue>

Making Permanent Repairs

But It Still Doesn't Work

RAID

RAID vergroten

De volgende stappen zijn nodig:

Na het wegvallen van 1 harddisk

Swap file on BTRFS

Test harddisk for bad blocks

Partitions-hupsie

Grub2: boot USB from GRUB prompt

Best way to use GRUB in a multiboot system

How to resue your system when the kernel won't boot

SUSE Broken? Don’t fear the chroot !

How to restore MBR on harddisk

How to resque a hanging linux

Hardware & Machines

Systems

Lap088

Akoya

Partitions akoya

Zbox

Partities

Hpspil

Partitions hpspil

Cpu3972

Samsie

Partitions samsie

WeTab

Partitions cpu3972

Amd64