It must be THE most annoying thing in the UNIX world : getting a file which has been created with Windows, or MS-DOS, and having these ^M end-of-line characters (which represents the odd carriage return characters in the Windows world)
Wikipedia speaks a better english than me :
The different newline conventions often cause text files that have been transferred between systems of different types to be displayed incorrectly. For example, files originating on Unix or Apple Macintosh systems may appear as a single long line on some Windows programs. Conversely, when viewing a file originating from a Windows computer on a Unix system, the extra CR may be displayed as ^M or <cr> at the end of each line or as a second line break.
# cat file_msdos.txt **this is a file generated with MSDOS***^M Visual Basic is fantastic^M All your base are belong to us^M ^M #
Try to outsmart your friends
Open the file with vi editor and write this sequence :
which must be read as « :%s/ »+ [ctrl+V] + [enter] + [enter]
# vi file_msdos.txt **this is a file generated with MSDOS***^M Visual Basic is fantastic^M All your base are belong to us^M ^M :%s/^M/
…which gives us :
**this is a file generated with MSDOS*** Visual Basic is fantastic All your base are belong to us :wq! #
Nice , isn’t it ?
Multiple files, multiple pains
If you need to script it because John-from-the-front-office sent you 1478 files with ^M characters, you may try the following, juste before sending your bloodlust ninjas to John’s house:
for file in $(ls *.txt) do cat $file | sed -e "s/\r\n//g" > $file done
(I assume the file is not too big, otherwise you may have to use a temporary file, then move the temp file to the original and replace it.)