January 26, 2012

Remote backup using ssh and tar

Filed under: Linux Reference — Paul Vint @ 1:17 pm

To backup a directory on a remote machine to a local machine (or the other way around) in Linux, there are several methods:

1. rsync -avz username@remoteserver.calm:/home/stuffToBackup /opt/backup
This will recursively copy everything from remote to local preserving permissions and ownership
2. scp -r -C username@remoteserver.calm:/home/stuffToBackup /opt/backup
Again, this will copy everything from remote to local, but there is one potential pitfall: It WILL follow symbolic links.
f.e.: If on the remote server there is a public_html directory, and a symlink www pointing to public_html, it will copy both as if they are regular directories.

Solution to the symlink problem? Don’t have rsync available?
Use ssh and tar:
ssh username@remoteserver.calm “tar cjp /home/stuffToBackup” | tar jxvp -C /opt/backup/
This will copy symlinks as-is (ie: create the symlink in the backup directory), using bzip2 compression (tar’s j option), preserving ownership and permissions (tar’s p option), and will extract the data to /opt/backup (tar’s -C option)

Unlike

March 31, 2011

rsync backups over ssh

Filed under: Linux Reference — Paul Vint @ 9:54 am

I always seem to forget how to do this, so here it is:



To copy files from /home/pvint/stuff directory to the /opt/backup directory on a remote server:

rsync -avz --progress /home/pvint/stuff -e ssh pvint@theBackupServer.calm:/opt/backup

Or going the other way, pull from remote server:

rsync -avz --progress -e ssh pvint@theBackupServer.calm:/opt/backup/stuff /home/pvint/

Notes:

  • The -a option is for "archive" and does a few things, like copies symlinks as symlinks, preserves owenership (only if you are running as root though!)
  • -v is for Verbose, -z compresses as it transfers
  • A note regarding trailing slashes: rsync -avz –progress /home/pvint/stuff -e ssh pvint@theBackupServer.calm:/opt/backup will copy the files to /opt/backup/stuff/files… whereas adding the trailing slash to /home/pvint/stuff/ will copy the files and directories in the 'stuff' directory directly to the /opt/backup directory (omitting adding the 'stuff' directory)
Unlike

March 9, 2011

500 – Intermittent Internal Server Error when posting form – Apache

Filed under: Linux Reference,Uncategorized — Paul Vint @ 1:49 pm

If you find that certain forms cause an “Internal Server Error” (500) in Apache on occasion, a good thing to check is for mod_security configuration blocking a “potential threat”.

In my case, I saw the following in /usr/local/apache/logs/modsec2_audit.log: 

[Wed Mar 09 13:36:34 2011] [error] [client 192.168.50.222] ModSecurity: Rule 7dd3610 [id "-"][file "/usr/local/apache/conf/modsec2/rootkits.conf"][line "155"] - Execution error - PCRE limits exceeded (-8): (null). [hostname "www.example.com"] [uri "/myFormScript.php"] [unique_id "AiGDV0PjjwgAAHwpOAoAAAAA"]
[Wed Mar 09 13:36:34 2011] [error] [client 192.168.50.222] ModSecurity: Access denied with code 500 (phase 2). Pattern match "((select|grant|delete|insert|drop|alter|replace|truncate|update|create|rename|describe)[[:space:]]+[A-Z|a-z|0-9|\\*| |\\,]+[[:space:]]+(from|into|table|database|index|view)[[:space:]]+[A-Z|a-z|0-9|\\*| |\\,]|UNION SELECT.*\\'.*\\'.*,[0-9].*INTO.*FROM)" at REQUEST_BODY. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "346"] [hostname "www.example.com"] [uri "/myFormScript.php"] [unique_id "AiGDV0PjjwgAAHwpOAoAAAAA"]

The solution (for me) for this problem was to whitelist my host (it’s an administrative tool in a CMS system, so it’s only used from certain IPs, so in /usr/local/apache/conf/modsec2/whitelist.conf I added the following line:

SecRule REMOTE_ADDR "^192.168.50.222$" noLog,allow

where 192.168.50.222 is the IP address of the machine that I access it from (not the server’s IP)

Unlike

March 7, 2011

Linux: rsync file copy and backup over ssh

Filed under: Linux Reference — Paul Vint @ 11:51 am

To copy files over the network from one machine to another:

Copy from remote server to local (pull):

rsync -avz -e ssh remoteuser@example.com:/home/source_directory /home/destination

Copy from local machine to remote hist (push):

rsync -avz /home/backups/source_directory -e sshremoteuser@example.com:/home/destination/directory

Unlike

March 1, 2011

Perl one-liner to find and replace text recursively

Filed under: Linux Reference — Paul Vint @ 2:27 pm

Handy little Perl one liner example to find and replace text across multiple filess/directories:

find . -name '*.php' -print0 | xargs -0 perl -pi -e 's/findThis/replaceWithThis/g'

Unlike

December 13, 2010

Resizing an LVM2 partition with ext4 filesystem

Filed under: Linux Reference — Paul Vint @ 6:56 am

To resize an ext4 formatted partition with LVM2 on a Linux system, it is recommended to unmount it, do an fsck, then resize it as per below, fsck again, then mount. Today, when my /usr/portage partition on my gentoo system was too full, I decided to try the “Damn the torpedoes, full speed ahead!” approach, and resize it online without unmounting, I simply did:
# lvextend -L +10G /dev/vg/portage
# resize2fs /dev/vg/portage
Note the “-L +10G”. The arguments to the -L option is the size to extend to, in this case, +10G means “add 10 Gig”. The suffix is Megabytes by default, and you can use K for kilobytes, M for megabytes, G
for gigabytes, T for terabytes, P for petabytes or E for exabytes, or explicitly specify the size without the +.

Unlike

December 10, 2010

Sendmail/procmail – not receiving email – “Suspicious rcfile” error

Filed under: Linux Reference — Tags: , , , , — Paul Vint @ 1:06 pm

Adding permissions on a directory containing a user’s .procmailrc file can cause failures in delivering mail, and you will see messages like the below in your log file (typically /var/log/maillog or similar):

Dec 9 03:16:04 host sendmail[2627]: oB98G4g2002627: forward /home/paul/.forward.host: Group writable directory
Dec 9 03:16:04 host sendmail[2627]: oB98G4g2002627: /home/paul: unsafe directory path, marked unsafe
Dec 9 03:16:04 host sendmail[2627]: oB98G4g2002627: forward /home/paul/.forward: Group writable directory
Dec 9 03:16:08 host procmail[2841]: Suspicious rcfile "/home/user/.procmailrc"
Dec 9 03:16:26 host procmail[3816]: Suspicious rcfile "/home/user/.procmailrc"

A quick peek at the source for procmail reveals the reasons that this may happen:

       /*
     * OK, so now we have opened an absolute rcfile, but for security
     * reasons we only accept it if it is owned by the recipient or
     * root and is not world writable, and the directory it is in is
     * not world writable or has the sticky bit set.  If this is the
     * default rcfile then we also outlaw group writability.
     */
{ register char*chp=lastdirsep(buf),c;
     c= *chp;
  if(((stbuf.st_uid!=uid&&stbuf.st_uid!=ROOT_uid||          /* check uid, */
          (stbuf.st_mode&S_IWOTH)||                   /* writable by others, */
          rctype==rct_DEFAULT&&            /* if the default then also check */
           (stbuf.st_mode&S_IWGRP)&&               /* for writable by group, */
           (NO_CHECK_stgid||stbuf.st_gid!=gid)
         )&&strcmp(devnull,buf)||            /* /dev/null is a special case, */
        (*chp='\0',stat(buf,&stbuf))||               /* check the directory, */
#ifndef CAN_chown                                  /* sticky and can't chown */
        !(stbuf.st_mode&S_ISVTX)&&                 /* means we don't care if */
#endif                                       /* it's group or world writable */
        ((stbuf.st_mode&(S_IWOTH|S_IXOTH))==(S_IWOTH|S_IXOTH)||
         rctype==rct_DEFAULT&&
          (stbuf.st_mode&(S_IWGRP|S_IXGRP))==(S_IWGRP|S_IXGRP)&&
          (NO_CHECK_stgid||stbuf.st_gid!=gid))))
      { *chp=c;
        goto suspicious_rc;

To summarize, procmail will abort delivery if:

  1. The procmailrc file is writeable by other users
  2. The directory conatining procmailrc is writeable by other users, or group writeable (and the sticky bit is not set on the directory)
  3. The procmailrc file is owned by either the recipient or by root
Unlike

December 9, 2010

Recursively remove files from CVS controlled directory

Filed under: Linux Reference — Paul Vint @ 2:40 pm

To recursively delete files in a directory tree that is controlled by CVS, at a Bash prompt do the following:
for dir in $(find -type d ! -name CVS); do for file in $(find $dir -maxdepth 1 -type f); do rm $file; cvs delete $file; done; done; cvs remove
Note that you still need to do a ‘cvs commit’.

Note that it won’t handle files with spaces or special characters nicely… but works for most purposes.

Unlike

October 14, 2010

Enabling spellcheck in OpenOffice 3 (Gentoo Linux)

Filed under: Linux Reference — Paul Vint @ 9:17 am

After installing OpenOffice 3.x in Gentoo Linux you may notice that the spellcheck doesn’t work (ie: you run spellcheck and it just says “Spell Check Complete”).
Assuming you have a dictionary installed (look in /usr/portage/app-dict and install the appropriate myspell-xx dictionary), you can enable the dictionary by going to “Tools” menu in OpenOffice, then select “Extension Manager”, select “Add” and browse to “/usr/lib/openoffice/share/extension/install and select the dictionary extension to add.
Restart OO, and it should work fine.

Unlike

September 2, 2010

Change the default “View Source” application in Firefox

Filed under: HTML and CSS,Linux Reference — Paul Vint @ 9:49 am

I find I’m constantly using the “View Source” option in Firefox while working on sites, but the default viewer isn’t the best.

To configure Firefox so it’ll use a different viewer, do the following:
1. In Firefox, enter “about:config” in the address bar (if you get a warning about dragons, heed it, but continue)
2. type “view” in the Filter field
3. Double-click on “view_source.editor.path” and enter the path to the editor you desire (I used /usr/bin/kate)
4. Double-click “view_source.editor.external” to set it to “True”
5. Restart Firefox

Unlike
Older Posts »

Powered by WordPress