...making Linux just a little more fun!
[ In reference to "2-Cent Tips" in LG#164 ]
Paul Sephton [paul at inet.co.za]
@$# This was originally entitled: "about 2c ext2 fragmentation" - Kat $#@
On Fri, 2009-07-03 at 22:46 +0200, Carlos Baiget wrote:
> another 0.5c: If any file in the given directory has a space in his > name, the defragmentation script will not work. To replace all spaces > inside the file name, i suggest to do a > > rename 's/\ /_/g' * > > previously or modify IFS environment variable to not consider space as > field separator. > > Carlos
Ah, yes; or simply quote the file names as follows:
fs=`echo "$line" | cut -f 1 -d':'`
fn=`echo "$line" | cut -f 2 -d':'`
# copy the file up to 10 times, preserving permissions
j=0;
while [ -f "$fn" -a $j -lt 10 ]; do
....
IFS also makes sense. Thanks for highlighting this rather serious error. I really should have taken the trouble to test this properly first!
Paul
Paul Sephton [paul at inet.co.za]
Aaaahhhh! Top posted! Too late, she cried!
[[[ It's okay, Paul - for you, I manually corrected it. Thanks for noticing and caring. -- Kat ]]]
Thomas Adam [thomas.adam22 at gmail.com]
2009/7/3 Paul Sephton <paul@inet.co.za>:
> Ah, yes; or simply quote the file names as follows: > > fs=`echo "$line" | cut -f 1 -d':'` > fn=`echo "$line" | cut -f 2 -d':'` > # copy the file up to 10 times, preserving permissions > j=0; > while [ -f "$fn" -a $j -lt 10 ]; do > .... > > IFS also makes sense. Thanks for highlighting this rather serious error. I > really should have taken the trouble to test this properly first! > > Paul > > On Fri, 2009-07-03 at 22:46 +0200, Carlos Baiget wrote: >> another 0.5c: If any file in the given directory has a space in his >> name, the defragmentation script will not work. To replace all spaces >> inside the file name, i suggest to do a >> >> rename 's/\ /_/g' *
Err, be careful here -- rename as a command is not the same across distributions. On Debian and I assume it's derivatives, it is indeed the classic perl script. On Redhat, it's something completely different. In which case "mmv" and friends can be used.
-- Thomas Adam
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 22:24 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <paul@inet.co.za>: > > On Fri, 2009-07-03 at 22:46 +0200, Carlos Baiget wrote: > >> another 0.5c: If any file in the given directory has a space in his > >> name, the defragmentation script will not work. To replace all spaces > >> inside the file name, i suggest to do a > >> > >> rename 's/\ /_/g' * > > Err, be careful here -- rename as a command is not the same across > distributions. On Debian and I assume it's derivatives, it is indeed > the classic perl script. On Redhat, it's something completely > different. In which case "mmv" and friends can be used. > > -- Thomas Adam
Good point. Anyway, the other problem with the script is that it needs root privilege to run '/sbin/filefrag'. The script is not really safe either as pointed out by Carlos. I have tested the following against a directory of files that contain spaces. On the other hand, the fact that 'filefrag' can't be run as a normal user rather limits it's use. chmod +s /sbin/filefrag is an option.
Can you spot any more loopholes?
#!/bin/sh
# Retrieve a list for fragmented files, #fragments:filename
FILEFRAG="/sbin/filefrag"
if [ ! -f $FILEFRAG ]; then
echo requires $FILEFRAG to defrag this directory
exit 0
fi
flist() {
for i in *; do
if [ -f "$i" ]; then
ff=`$FILEFRAG "$i"`
fn=`echo "$ff" | cut -f1 -d':'`
fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '`
if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi
fi
done
}
IFS='
'
# Sort the list numeric, descending
flist | sort -n -r |
(
# for each file
while read line; do
fs=`echo "$line" | cut -f 1 -d':'`
fn=`echo "$line" | cut -f 2 -d':'`
# copy the file up to 10 times, preserving permissions
j=0;
while [ -f "$fn" -a $j -lt 10 ]; do
j=$[ $j + 1 ]
TMP=$$.tmp.$j
if ! cp -p "$fn" "$TMP"; then
echo copy failed [$fn]
j=10
else
# test the new temp file's fragmentation, and if less than the
# original, move the temp file over the original
ns=`$FILEFRAG $TMP | cut -f2 -d':' | cut -f2 -d' '`
if [ $ns -lt $fs ]; then
mv "$TMP" "$fn"
fs=$ns
if [ $ns -lt 2 ]; then j=10; fi
fi
fi
done
j=0;
# clean up temporary files
while [ $j -lt 10 ]; do
j=$[ $j + 1 ]
TMP=$$.tmp.$j
if [ -f "$TMP" ]; then
rm "$TMP"
else
j=10
fi
done
done
)
# report fragmentation
for i in *; do if [ -f $i ]; then $FILEFRAG $i; fi; done
Thomas Adam [thomas.adam22 at gmail.com]
2009/7/3 Paul Sephton <paul@inet.co.za>:
> Can you spot any more loopholes?
I scanned it very quickly, you, like everyone else, needs to "USE" ""MORE"" """""QUOTES""""".
I can't stress enough just how important that is.
-- Thomas Adam
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 22:54 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <paul@inet.co.za>: > > Can you spot any more loopholes? > > I scanned it very quickly, you, like everyone else, needs to "USE" > ""MORE"" """""QUOTES""""". > > I can't stress enough just how important that is. > > -- Thomas Adam
... as in the last line,
for i in *; do if [ -f $i ]; then $FILEFRAG $i; fi; done should be for i in *; do if [ -f "$i" ]; then $FILEFRAG "$i"; fi; done
oh well....
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 22:54 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <paul@inet.co.za>: > > Can you spot any more loopholes? > > I scanned it very quickly, you, like everyone else, needs to "USE" > ""MORE"" """""QUOTES""""". > I can't stress enough just how important that is. > -- Thomas Adam
Ok, here's the latest. Gives a bit more feedback about what is happening in a more readable way. Also fixes a bug that left temp files lying around. I'm running out of still fragmented directories/files to test it on.
#!/bin/sh
# Retrieve a list for fragmented files, #fragments:filename
FILEFRAG="/sbin/filefrag"
if [ ! -f $FILEFRAG ]; then
echo requires $FILEFRAG to defrag this directory
exit 0
fi
flist() {
for i in *; do
if [ -f "$i" ]; then
ff=`$FILEFRAG "$i"`
fn=`echo "$ff" | cut -f1 -d':'`
fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '`
if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi
fi
done
}
# Internal Field Separator is now a newline
IFS='
'
# Sort the list numeric, descending
echo Reading files...
flist | sort -n -r |
(
# for each file
while read line; do
fs=`echo "$line" | cut -f 1 -d':'`
fn=`echo "$line" | cut -f 2 -d':'`
# copy the file up to 10 times, preserving permissions
j=0;
echo -e -n "$fn\t$fs"
while [ -f "$fn" -a $j -lt 10 ]; do
j=$[ $j + 1 ]
echo -n "."
TMP=$$.tmp.$j
if ! cp -p "$fn" "$TMP"; then
echo copy failed [$fn]
j=10
else
# test the new temp file's fragmentation, and if less than the
# original, move the temp file over the original
ns=`$FILEFRAG $TMP | cut -f2 -d':' | cut -f2 -d' '`
if [ $ns -lt $fs ]; then
mv "$TMP" "$fn"
fs=$ns
echo -n "$fs"
if [ $ns -lt 2 ]; then j=10; fi
fi
fi
done
echo
echo " - file now has $fs fragments"
j=0;
# clean up temporary files
while [ $j -lt 10 ]; do
j=$[ $j + 1 ]
TMP=$$.tmp.$j
if [ -f "$TMP" ]; then
rm "$TMP"
fi
done
done
)
Thomas Adam [thomas.adam22 at gmail.com]
2009/7/3 Paul Sephton <paul@inet.co.za>:
> if [ ! -f $FILEFRAG ]; then
[ -x "$FILEFRAG" ]
Just on the off-chance the file isn't executable for some odd reason.
> echo requires $FILEFRAG to defrag this directory
echo "requires $FILEFRAG to defrag this directory"
> exit 0
> fi
>
> flist() {
> for i in *; do
> if [ -f "$i" ]; then
> ff=`$FILEFRAG "$i"`
To be honest, only the most ancient of shells, and I mean ancient, won't understand:
$()
For command substitution, over backticks. Given nesting backticks is problematic, you should just use "$()".
> fn=`echo "$ff" | cut -f1 -d':'` > fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '` > if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi
Odd -- you claim "/bin/sh", yet "echo -e" isn't POSIX -- and why is it even invoked as such here, when you're not asking for backslash interpretation?
> echo -e -n "$fn\t$fs"
printf()
> j=$[ $j + 1 ]
((J++)) -- the square-brackets are an arcane syntax.
-- Thomas Adam
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 23:50 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <paul@inet.co.za>: > > if [ ! -f $FILEFRAG ]; then > > ``` > [ -x "$FILEFRAG" ] > ''' > > Just on the off-chance the file isn't executable for some odd reason.
Cool. I had changed it to -x elsewhere, but the real problem is if the script is run as an ordinary user, the execution fails because of a kernel access restriction. I was thinking of checking
-x $FILEFRAG -a $USER=root or -x $FILEFRAG -a -s $FILEFRAG. Might be better.
> > echo requires $FILEFRAG to defrag this directory
>
> ```
> echo "requires $FILEFRAG to defrag this directory"
>
> > exit 0
> > fi
> >
> > flist() {
> > for i in *; do
> > if [ -f "$i" ]; then
> > ff=`$FILEFRAG "$i"`
>
> To be honest, only the most ancient of shells, and I mean ancient,
> won't understand:
>
> ```
> $()
> '''
... you can tell I'm rather ancient, can't you? I have never used $() before, other than in makefiles...
> For command substitution, over backticks. Given nesting backticks is > problematic, you should just use "$()". > > > fn=`echo "$ff" | cut -f1 -d':'` > > fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '` > > if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi > > Odd -- you claim "/bin/sh", yet "echo -e" isn't POSIX -- and why is it > even invoked as such here, when you're not asking for backslash > interpretation? > > > echo -e -n "$fn\t$fs" > > printf()
printf() is POSIX? I didn't know. Agreed, echo -e is unnecessary in the first instance. I'll just make it #bin/bash rather, or as you suggest, try printf() instead.
> > > j=$[ $j + 1 ] > > ((J++)) -- the square-brackets are an arcane syntax.
I'm not just ancient, but a bit arcane too 
> -- Thomas Adam
Thanks, Thomas. Your help is appreciated.
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 23:50 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <paul@inet.co.za>: > > if [ ! -f $FILEFRAG ]; then > > ``` > [ -x "$FILEFRAG" ] > ''' > > Just on the off-chance the file isn't executable for some odd reason.
<snip> Ok, this actually reads and works a whole lot better. Thanks for the tips, Thomas:
#!/bin/sh
# Retrieve a list for fragmented files, #fragments:filename
FILEFRAG="/sbin/filefrag"
if [ ! -x $FILEFRAG ]; then
echo requires $FILEFRAG to defrag this directory
exit 0
fi
if [ ! "$USER" = "root" ]; then
if [ ! -u $FILEFRAG ]; then
echo $FILEFRAG can only be run as root, or should be SUID root
exit 0
fi
fi
flist() {
for i in *; do
if [ -f "$i" ]; then
ff=$( $FILEFRAG "$i" )
fn=$( echo "$ff" | cut -f1 -d':' )
fs=$( echo "$ff" | cut -f2 -d':' | cut -f2 -d' ' )
if [ -f "$fn" -a $fs -gt 1 ]; then printf "$fs:$fn\n"; fi
fi
done
}
# Internal Field Separator is now a newline
IFS='
'
# Sort the list numeric, descending
echo Reading files...
flist | sort -n -r |
(
# for each file
while read line; do
fs=$( echo "$line" | cut -f 1 -d':' )
fn=$( echo "$line" | cut -f 2 -d':' )
# copy the file up to 10 times, preserving permissions
j=0
printf "$fn\t$fs"
while [ -f "$fn" -a $j -lt 10 ]; do
((j++))
printf "."
TMP=$$.tmp.$j
if [ -f "$TMP" ]; then
echo "Error! target temp file already exists!"
elif ! cp -p "$fn" "$TMP"; then
echo copy failed [$fn]
j=10
else
# test the new temp file's fragmentation, and if less than the
# original, move the temp file over the original
ns=$( $FILEFRAG "$TMP" | cut -f2 -d':' | cut -f2 -d' ' )
if [ $ns -lt $fs ]; then
mv "$TMP" "$fn"
fs=$ns
printf "$fs"
if [ $ns -lt 2 ]; then j=10; fi
fi
fi
done
echo
echo " (file now has $fs fragments)"
j=0
# clean up temporary files
while [ $j -lt 10 ]; do
((j++))
TMP=$$.tmp.$j
if [ -f "$TMP" ]; then
rm "$TMP"
fi
done
done
)