Questions tagged [files]
Organized data encoded in a pre-defined format and stored on a filesystem.
684
questions
339
votes
18
answers
442k
views
Copying a large directory tree locally? cp or rsync?
I have to copy a large directory tree, about 1.8 TB. It's all local. Out of habit I'd use rsync, however I wonder if there's much point, and if I should rather use cp.
I'm worried about permissions ...
120
votes
8
answers
187k
views
How do I create a symbolic link in Windows?
Windows Vista added the ability to create symbolic links to files and directories. How do I create a symbolic link and what are the current consumer and server versions of Windows that support it?
103
votes
4
answers
90k
views
Perform rsync while following sym links
I have a directory that contains symbolic links to other directories located on different media on my system:
/opt/lun1/2011
/opt/lun1/2010
/opt/lun2/2009
/opt/lun2/2008
/opt/lun3/2007
But the ...
101
votes
10
answers
90k
views
How do I do Multihop SCP transfers between machines?
I want to copy a file from my machine A to server C, but only have access to server C through server B.
Instead of first transferring to server B, log in and then transfer to server C, Is is possible ...
93
votes
9
answers
44k
views
Linux - Is there a way to prevent/protect a file from being deleted even by root?
I have a very important file which an application in my workplace uses, i need to make sure it is not delete whatsoever, how can I do that?
84
votes
9
answers
109k
views
How to make scp copy hidden files?
I often use SCP to copy files around - particularly web-related files. The problem is that whenever I do this, I can't get my command to copy hidden files (eg, .htaccess).
I typically invoke this:
...
79
votes
3
answers
25k
views
In Bash, are wildcard expansions guaranteed to be in order?
Is the expansion of a wildcard in Bash guaranteed to be in alphabetical order? I am forced to split a large file into 10 Mb pieces so that they can be be accepted by my Mercurial repository.
So I was ...
66
votes
7
answers
135k
views
Extracting files from Clonezilla images
Is there a way to browse Clonezilla images and extract individual files from them without restoring the whole image?
60
votes
18
answers
30k
views
Favorite rsync tips and tricks
The more I use rsync the more I realise that it's a swiss army knife of file transfer. There are so many options. I recently found out that you can go --remove-source-files and it'll delete a file ...
53
votes
6
answers
67k
views
How to move files between two S3 buckets with minimum cost?
I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone.
How could I ...
48
votes
2
answers
78k
views
nginx split large configuration file
My nginx default configuration file is becoming huge. I'd like to split it to smaller config files, each including only one, maximum 4 locations to each file, so that I can enable/disable them quickly....
47
votes
10
answers
154k
views
Linux (mv or cp) specific files from a text list of files?
I have a directory of many files, something like 50,000 pdf's and other files on a server. I need to move specific ones to another directory. I can generate a list of the files that need to be moved ...
46
votes
2
answers
65k
views
How to delete .fuse_hidden* files?
I have small Linux server (Debian Squeeze) which runs a Samba server which is configured to share some folders with some windows machines. While trying to delete one of the directories from windows I ...
44
votes
6
answers
220k
views
Move files to another directory which are older than a date
I am looking for a solution to move files that are year older from today. My log partition is getting full, but I can not remove them. They are needed for a long long time. Anyway one solution I came ...
42
votes
3
answers
119k
views
How to view hidden files using Linux `find` command
On a Linux server, I need to find all files with a certain file extension in the current directory and all sub-directories.
Previously, I have always used the following command:
find . -type f | grep -...
41
votes
6
answers
136k
views
How to rename multiple files by replacing word in file name?
Replace ACDC to AC-DC
For example we have these files
ACDC - Rock N' Roll Ain't Noise Pollution.xxx
ACDC - Rocker.xxx
ACDC - Shoot To Thrill.xxx
I want them to become:
AC-DC - Rock N' Roll Ain't ...
39
votes
3
answers
57k
views
Does the Midnight Commander have an option to display actual dir sizes?
i have plenty of empty directories and i wonder if there's a way to display actual direcory sizes (after some sort of scan maybe) in MC
35
votes
6
answers
29k
views
Merge 2 directory trees in Linux without copying?
I have two directory trees with similar layouts, i.e.
.
|-- dir1
| |-- a
| | |-- file1.txt
| | `-- file2.txt
| |-- b
| | `-- file3.txt
| `-- c
| `-- file4.txt
`-- ...
33
votes
9
answers
50k
views
Best way to compare (diff) a full directory structure?
What's the best way to compare directory structures?
I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
28
votes
4
answers
40k
views
How to delete all hidden files and directories using Bash?
The obvious solution produces an exit code of 1:
bash$ rm -rf .*
rm: cannot remove directory `.'
rm: cannot remove directory `..'
bash$ echo $?
1
One possible solution will skip the "." and ".."
...
28
votes
1
answer
28k
views
Is it possible to configure the location of "Temporary ASP.NET Files" folder in IIS?
Under IIS7.5 after switching pool identity to domain user I got this error:
"The current identity (domain\username) does not have write access to 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\...
27
votes
9
answers
61k
views
How to get IIS7 to release a locked file?
During our production builds, a very large (10 megabyte) static content file in the root directory will sometimes be locked by IIS and cannot be deleted by the clean task. This is presumably because ...
26
votes
7
answers
76k
views
How to deny the web access to some files?
I need to do an operation a bit strange.
First, i run on Debian, apache2 (which 'runs' as user www-data)
So, I have simple text file with .txt ot .ini, or whatever extension, doesnt matter.
These ...
24
votes
10
answers
56k
views
Copy large file from one Linux server to another
I'm attempting to copy a 75 gigabyte tgz (mysql lvm snapshot) from a Linux server in our LA data center to another Linux server in our NY data center over a 10MB link.
I am getting about 20-30Kb/s ...
24
votes
6
answers
1k
views
How do I prove two files are the same legally?
We had someone steal some files before quitting and it has eventually come down to a lawsuit. I've now been provided with a cd of files and I have to "prove" that they are our files by matching them ...
24
votes
4
answers
5k
views
Pushing Large Files to 500+ Computers
I work with a team to manage 500-600 rented Windows 7 computers for an annual conference. We have a large amount of data that needs to be synced to these computers, up to 1 TiB. The computers are ...
23
votes
6
answers
51k
views
Recursively copying hidden files - Linux
Is there an easy way to recursively copy all hidden files in a directory to another directory? I would like to back up just all the settings files in a home directory, not the normal files. I tried:
...
22
votes
8
answers
23k
views
Doing an rm -rf on a massive directory tree takes hours
We're using rsnapshot for backups. It keeps lots of snapshots of the backuped up file, but it does delete old ones. This is good. However it's taking about 7 hours to do a rm -rf on a massive ...
22
votes
1
answer
59k
views
Optimizing rsync for transferring huge files quickly
This is what I need:
Rsync daemon running on source server. Destination server does a rsync pull as below:
rsync -ptv rsync://source_ip:document/source_path/*.abc destination_path/
Huge binary files ...
22
votes
4
answers
21k
views
Rsync show progress for individual file
I'm using rsync to upload/download a pretty large file (20Gb). I'm aware of --progress option, but what it does in my case is just showing me the result at the end.
My problem is that during some ...
22
votes
2
answers
62k
views
What is stored in %Windir%\System32\LogFiles\WMI\RtBackup?
I occasionally notice in Resource Monitor hard disk activity related to ETL files in the folder C:\Windows\System32\LogFiles\WMI\RtBackup.
Which process/service creates these ETL files and what is ...
21
votes
9
answers
36k
views
Get total files size from a file containing a file list
I have a file containing a list of files that I would like to know the total files size. Is there a command to do so?
My OS is a very basic linux (Qnap TS-410).
EDIT:
A few lines from the file:
/...
20
votes
7
answers
4k
views
Linux mv file with long name
In Linux I sometimes rename files like this:
mv dir1/dir2/dir3/file.txt dir1/dir2/dir3/file.txt.old
Note that I want to just rename the file, not move it to another directory.
Is there a command ...
19
votes
9
answers
32k
views
Is it possible to hide lost+found?
Given a partition intended solely for storing music, video and so-on, is it possible to hide the lost+found directory?
19
votes
1
answer
21k
views
What is fastest way to copy a sparse file? What method results in the smallest file?
BACKGROUND: I'm copying a sparse qcow2 VM image that is 200GB in size, but has 16GB of allocated blocks. I've tried various methods to copy this sparse file within the same server and have some ...
18
votes
2
answers
53k
views
nginx uLimit 'worker_connections exceed open file resource limit: 1024'
I keep getting this error in nginx/error.log and its driving me nuts:
8096 worker_connections exceed open file resource limit: 1024
I've tried everything I can think of and cant figure out what is ...
18
votes
3
answers
9k
views
Building a 1,000M row MySQL table
This question is reposted from Stack Overflow based on a suggestion in the comments, apologies for the duplication.
Questions
Question 1: as the size of the database table gets larger how can I tune ...
16
votes
7
answers
13k
views
In *nix, how to determine which filesystem a particular file is on?
In a generic, modern unix environment (say, GNU/Linux, GNU/Solaris, or Mac OS X), is there a good way to determine which mountpoint and filesystem-type a particular absolute file path is on?
I ...
16
votes
3
answers
51k
views
Is there a hard limit to the number of files a directory can have? [duplicate]
Wondering if there is a limit to the number of files that can be stored inside a directory, in CentOS 6. There is one particular directory which could potentially have millions of subdirectories.
...
16
votes
6
answers
9k
views
Files disappearing on linux server
I've got 4 specific files that seem to keep disappearing from a user's home directory. As far as we know, there are no cronjobs or other automated tasks that would be removing them. I've setup auditd ...
16
votes
4
answers
3k
views
Concatenating files to a virtual file on Linux
On a Linux system, is there any way to concatenate a series of files into one exposed file for reading and writing while not actually taking up another N bytes of disk space? I was hoping for ...
15
votes
6
answers
48k
views
Moving a Logical Volume directly from one server to another over the network?
I have a KVM host machine with several VMs on it. Each VM uses a Logical Volume on the host. I need to copy the LVs to another host machine.
Normally, I would use something like:
dd if=/the/logical-...
15
votes
7
answers
68k
views
How to Find a file in FreeBSD
I'm trying to locate all copies of example.filename on my FreeBSD server. What's the best / easiest / most efficient way to do this?
15
votes
4
answers
8k
views
read specified range of lines from a file
I have i file that contains 100000 line
how i can get the lines from line# 5555 to line# 7777 under linux.
Thanks for all.
14
votes
19
answers
28k
views
What is the simplest and fastest way to transfer large file through a Windows network?
I have a Window Server 2000 machine running MS SQL Server that stores over 20GB of data. The database is backed-up every day to the second harddrive. I want to transfer those backup files to another ...
14
votes
3
answers
31k
views
Systemd drop-in fails to create PID file
I have a drop-in for systemd-machined at the path /etc/systemd/system/systemd-machined.service.d/10-machined-pid-file.conf. when I run systemctl status systemd-machined I do see the lines
Drop-In: /...
14
votes
4
answers
27k
views
Monitor open process files on Linux in real-time
The files opened by a process can be found with this command:
ls -l /proc/PID/fd
Is there any way that can be done in a more interactive way like tail, auto-refreshing every x seconds?
14
votes
7
answers
26k
views
How long do you keep log files?
I have an application which writes its log files in a special folder. Now I'd like to add a functionality to delete these logs after a defined period of time automatically. But how long should I keep ...
14
votes
6
answers
36k
views
Best method to track folder size growth over time?
I have a file server that has a pretty large folder tree. There's a shared folder, under that are 5 departmental folders. Nested inside of those are thousands of subfolders and files.
I would like ...
13
votes
6
answers
56k
views
How do I delete a locked file in Windows?
I'm moving some files around on the server (Windows Server 2003), and there's a locked file that no one seems to have open. How do I delete this file?