Merge pull request #3 from ehanson8/master

Updating scripts

Author: ehanson8

README.md

@@ -2,19 +2,22 @@
 Scripts for moving files and renaming directories
 
 #### [collapseDirectories.py](/collapseDirectories.py)
-Based on the file path entered, this script moves any file existing below the eighth level folder in a Linux directory structure (i.e. /media/bitCurator/Storage/Collection/IncomingTransfer/BatchNumber/Disc/JobName) up to the eigth level folder. The script writes a log of all files that were moved, including their old location and their new location.
+Based on the file path entered, moves any file existing below the eighth level folder in a Linux directory structure (i.e. /media/bitCurator/Storage/Collection/IncomingTransfer/BatchNumber/Disc/JobName) up to the eigth level folder. The script writes a log of all files that were moved, including their old location and their new location.
 
-#### [extractFolderNames.py](/extractFilenames.py)
-Based on the file path entered, this script extracts the names of all nested folders and writes them to a CSV file.
+#### [compareFilesInTwoDirectories.py](/compareFilesInTwoDirectories.py.py)
+Based on the primary and secondary file paths entered, produces two CSV files that list the files that exist in one directory but are missing from other directory.
 
 #### [extractFilenames.py](/extractFilenames.py)
-Based on the file path entered and file extension specified, this script extracts the names of all relevant files in the directory and writes them to a CSV file. Additionally, you can specify a prefix to be added to the edited file name in the "newFile" column.  Additionally, this script creates a log of all files, whether they have the specified extension or not, in the directory for troubleshooting potential problems.
+Based on the file path entered and file extension specified, extracts the names of all relevant files in the directory and writes them to a CSV file. Additionally, you can specify a prefix to be added to the edited file name in the "newFile" column.  Additionally, this script creates a log of all files, whether they have the specified extension or not, in the directory for troubleshooting potential problems.
 
-#### [renameFiles.py](/renameFiles.py)
-Based on the file path entered, this script renames files in the directory according to a specified CSV file with the columns named 'file' and 'newFile,' provided that there is a match between the files and the names in the 'file' column. The script writes a log of all files that were renamed, including their old name and their new name.
+#### [extractFolderNames.py](/extractFilenames.py)
+Based on the file path entered, extracts the names of all nested folders and writes them to a CSV file.
 
 #### [renameDirectories.py](/renameDirectories.py)
-Based on the file path entered, this script renames all nested folders according to a CSV file named 'FolderNames.csv' with the columns named 'oldFolder' and 'newFolder,' provided that there is a match between the nested folders and the names in the 'oldFolder' column. The script writes a log of all folders that were renamed, including their old name and their new name.
+Based on the file path entered, renames all nested folders according to a CSV file named 'FolderNames.csv' with the columns named 'oldFolder' and 'newFolder,' provided that there is a match between the nested folders and the names in the 'oldFolder' column. The script writes a log of all folders that were renamed, including their old name and their new name.
+
+#### [renameFiles.py](/renameFiles.py)
+Based on the file path entered, renames files in the directory according to a specified CSV file with the columns named 'file' and 'newFile,' provided that there is a match between the files and the names in the 'file' column. The script writes a log of all files that were renamed, including their old name and their new name. The user will be prompted as to whether they want the file name changes executed or whether they just want log of the expected file name changes.
 
 #### [samplingScript.py](/samplingScript.py)
-Based on the file path entered, the script selects every 10th file of a folder (after skipping the first file) and moves it over identically named folder in a 'sampled' directory while the skipped files are moved into an identically named folder in an 'unsampled' directory. The script also writes 'sampled' and 'unsampled' logs of all files that were moved, including their old location and their new location.
+Based on the file path entered, selects every 10th file of a folder (after skipping the first file) and moves it over identically named folder in a 'sampled' directory while the skipped files are moved into an identically named folder in an 'unsampled' directory. The script also writes 'sampled' and 'unsampled' logs of all files that were moved, including their old location and their new location.

compareFilesInTwoDirectories.py

@@ -0,0 +1,49 @@
+import os
+import csv
+from datetime import datetime
+import time
+
+parser = argparse.ArgumentParser()
+parser.add_argument('-1', '--primaryFilePath', help='the primary file path (C:/Test/). optional - if not provided, the script will ask for input')
+parser.add_argument('-2', '--secondaryFilePath', help='the secondary file path (C:/Test/). optional - if not provided, the script will ask for input')
+args = parser.parse_args()
+
+if args.primaryFilePath:
+    key = args.primaryFilePath
+else:
+    primaryFilePath = raw_input('Enter primary file path (C:/Test/): ')
+if args.secondaryFilePath:
+    key2 = args.secondaryFilePath
+else:
+    secondaryFilePath = raw_input('Enter secondary file path (C:/Test/): ')
+
+startTime = time.time()
+f=csv.writer(open('filesMissingFromSecondaryDirectory'+datetime.now().strftime('%Y-%m-%d %H.%M.%S')+'.csv','wb'))
+f.writerow(['directory']+['file'])
+f2=csv.writer(open('filesMissingFromPrimaryDirectory'+datetime.now().strftime('%Y-%m-%d %H.%M.%S')+'.csv','wb'))
+f2.writerow(['directory']+['file'])
+primaryFilePathDict = {}
+for root, dirs, files in os.walk(primaryFilePath, topdown=True):
+    for file in files:
+        primaryFilePathDict[file] = root
+secondaryFilePathDict = {}
+for root, dirs, files in os.walk(secondaryFilePath, topdown=True):
+    for file in files:
+        secondaryFilePathDict[file] = root
+
+for k, v in primaryFilePathDict.items():
+    if k in secondaryFilePathDict:
+        pass
+    else:
+        f.writerow([v]+[k])
+
+for k, v in secondaryFilePathDict.items():
+    if k in primaryFilePathDict:
+        pass
+    else:
+        f2.writerow([v]+[k])
+
+elapsedTime = time.time() - startTime
+m, s = divmod(elapsedTime, 60)
+h, m = divmod(m, 60)
+print 'Total script run time: ', '%d:%02d:%02d' % (h, m, s)

extractFilenames.py

@@ -6,6 +6,7 @@
 fileExtension = raw_input('Enter file extension: ')
 prefix = raw_input('Enter prefix to be added: ')
 
+startTime = time.time()
 f = csv.writer(open('fileListing.csv', 'wb'))
 f.writerow(['file'] + ['newFile'])
 f2 = csv.writer(open('fullfileListing.csv', 'wb'))
@@ -14,7 +15,7 @@
 for root, dirs, files in directories:
     for file in files:
         f2.writerow([file])
-        if file[-3:] == fileExtension
+        if file[-3:] == fileExtension:
             itemID = file[0:5]
             newFile = prefix + itemID + fileExtension
             f.writerow([file] + [newFile])

renameFiles.py

@@ -2,14 +2,31 @@
 import csv
 from datetime import datetime
 import time
+import argparse
 
-filePath = raw_input('Enter file path (C:/Test/): ')
-nameChangeFile = raw_input('Enter name of CSV with name changes: ')
+parser = argparse.ArgumentParser()
+parser.add_argument('-d', '--directory', help='the directory of the files to be renamed. optional - if not provided, the script will ask for input')
+parser.add_argument('-f', '--fileName', help='the CSV file of name changes. optional - if not provided, the script will ask for input')
+parser.add_argument('-m', '--makeChanges', help='Enter "true" to if the script should actually rename the files (otherwise, it will only create a log of the expected file name changes). optional - if not provided, the script will to "false"')
+args = parser.parse_args()
+
+if args.directory:
+    directory = args.directory
+else:
+    directory = raw_input('Enter the directory of the files to be renamed: ')
+if args.fileName:
+    fileName = args.fileName
+else:
+    fileName = raw_input('Enter the CSV file of name changes (including \'.csv\'): ')
+if args.makeChanges:
+    makeChanges = args.makeChanges
+else:
+    makeChanges = raw_input('Enter "true" to if the script should actually rename the files (otherwise, it will only create a log of the expected file name changes): ')
 
 startTime = time.time()
 f=csv.writer(open('renameLog'+datetime.now().strftime('%Y-%m-%d %H.%M.%S')+'.csv','wb'))
 f.writerow(['oldFilename']+['newFilename'])
-for root, dirs, files in os.walk(filePath, topdown=True):
+for root, dirs, files in os.walk(directory, topdown=True):
     for file in files:
         with open(nameChangeFile) as csvfile:
             reader = csv.DictReader(csvfile)
@@ -21,8 +38,10 @@
                     oldPath = os.path.join(root,file)
                     newPath = os.path.join(root,newFilename)
                     f.writerow([oldPath]+[newPath])
-                    #Uncomment the following line to acutally rename files rather than just writing a log file of the changes to be made
-                    #os.rename(oldPath,newPath)
+                    if makeChanges == 'true'
+                        os.rename(oldPath,newPath)
+                    else:
+                        print 'log of expected file name changes created only, no files renamed'
 
 elapsedTime = time.time() - startTime
 m, s = divmod(elapsedTime, 60)