Questions tagged [files]
Organized data encoded in a pre-defined format and stored on a filesystem.
684
questions
339
votes
18
answers
442k
views
Copying a large directory tree locally? cp or rsync?
I have to copy a large directory tree, about 1.8 TB. It's all local. Out of habit I'd use rsync, however I wonder if there's much point, and if I should rather use cp.
I'm worried about permissions ...
- 32.3k
120
votes
8
answers
187k
views
How do I create a symbolic link in Windows?
Windows Vista added the ability to create symbolic links to files and directories. How do I create a symbolic link and what are the current consumer and server versions of Windows that support it?
- 3,530
103
votes
4
answers
90k
views
Perform rsync while following sym links
I have a directory that contains symbolic links to other directories located on different media on my system:
/opt/lun1/2011
/opt/lun1/2010
/opt/lun2/2009
/opt/lun2/2008
/opt/lun3/2007
But the ...
- 2,232
101
votes
10
answers
90k
views
How do I do Multihop SCP transfers between machines?
I want to copy a file from my machine A to server C, but only have access to server C through server B.
Instead of first transferring to server B, log in and then transfer to server C, Is is possible ...
- 1,111
93
votes
9
answers
44k
views
Linux - Is there a way to prevent/protect a file from being deleted even by root?
I have a very important file which an application in my workplace uses, i need to make sure it is not delete whatsoever, how can I do that?
user256033
84
votes
9
answers
109k
views
How to make scp copy hidden files?
I often use SCP to copy files around - particularly web-related files. The problem is that whenever I do this, I can't get my command to copy hidden files (eg, .htaccess).
I typically invoke this:
...
- 1,148
79
votes
3
answers
25k
views
In Bash, are wildcard expansions guaranteed to be in order?
Is the expansion of a wildcard in Bash guaranteed to be in alphabetical order? I am forced to split a large file into 10 Mb pieces so that they can be be accepted by my Mercurial repository.
So I was ...
- 937
66
votes
7
answers
135k
views
Extracting files from Clonezilla images
Is there a way to browse Clonezilla images and extract individual files from them without restoring the whole image?
- 2,773
60
votes
18
answers
30k
views
Favorite rsync tips and tricks
The more I use rsync the more I realise that it's a swiss army knife of file transfer. There are so many options. I recently found out that you can go --remove-source-files and it'll delete a file ...
- 32.3k
53
votes
6
answers
67k
views
How to move files between two S3 buckets with minimum cost?
I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone.
How could I ...
- 823
48
votes
2
answers
78k
views
nginx split large configuration file
My nginx default configuration file is becoming huge. I'd like to split it to smaller config files, each including only one, maximum 4 locations to each file, so that I can enable/disable them quickly....
- 614
47
votes
10
answers
154k
views
Linux (mv or cp) specific files from a text list of files?
I have a directory of many files, something like 50,000 pdf's and other files on a server. I need to move specific ones to another directory. I can generate a list of the files that need to be moved ...
- 643
46
votes
2
answers
65k
views
How to delete .fuse_hidden* files?
I have small Linux server (Debian Squeeze) which runs a Samba server which is configured to share some folders with some windows machines. While trying to delete one of the directories from windows I ...
- 615
44
votes
6
answers
220k
views
Move files to another directory which are older than a date
I am looking for a solution to move files that are year older from today. My log partition is getting full, but I can not remove them. They are needed for a long long time. Anyway one solution I came ...
- 988
42
votes
3
answers
119k
views
How to view hidden files using Linux `find` command
On a Linux server, I need to find all files with a certain file extension in the current directory and all sub-directories.
Previously, I have always used the following command:
find . -type f | grep -...
- 4,367
41
votes
6
answers
136k
views
How to rename multiple files by replacing word in file name?
Replace ACDC to AC-DC
For example we have these files
ACDC - Rock N' Roll Ain't Noise Pollution.xxx
ACDC - Rocker.xxx
ACDC - Shoot To Thrill.xxx
I want them to become:
AC-DC - Rock N' Roll Ain't ...
- 1,584
39
votes
3
answers
57k
views
Does the Midnight Commander have an option to display actual dir sizes?
i have plenty of empty directories and i wonder if there's a way to display actual direcory sizes (after some sort of scan maybe) in MC
- 391
35
votes
6
answers
29k
views
Merge 2 directory trees in Linux without copying?
I have two directory trees with similar layouts, i.e.
.
|-- dir1
| |-- a
| | |-- file1.txt
| | `-- file2.txt
| |-- b
| | `-- file3.txt
| `-- c
| `-- file4.txt
`-- ...
- 613
33
votes
9
answers
50k
views
Best way to compare (diff) a full directory structure?
What's the best way to compare directory structures?
I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source ...
- 13.6k
28
votes
4
answers
40k
views
How to delete all hidden files and directories using Bash?
The obvious solution produces an exit code of 1:
bash$ rm -rf .*
rm: cannot remove directory `.'
rm: cannot remove directory `..'
bash$ echo $?
1
One possible solution will skip the "." and ".."
...
- 475
28
votes
1
answer
28k
views
Is it possible to configure the location of "Temporary ASP.NET Files" folder in IIS?
Under IIS7.5 after switching pool identity to domain user I got this error:
"The current identity (domain\username) does not have write access to 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\...
- 383
27
votes
9
answers
61k
views
How to get IIS7 to release a locked file?
During our production builds, a very large (10 megabyte) static content file in the root directory will sometimes be locked by IIS and cannot be deleted by the clean task. This is presumably because ...
- 685
26
votes
7
answers
76k
views
How to deny the web access to some files?
I need to do an operation a bit strange.
First, i run on Debian, apache2 (which 'runs' as user www-data)
So, I have simple text file with .txt ot .ini, or whatever extension, doesnt matter.
These ...
- 487
24
votes
10
answers
56k
views
Copy large file from one Linux server to another
I'm attempting to copy a 75 gigabyte tgz (mysql lvm snapshot) from a Linux server in our LA data center to another Linux server in our NY data center over a 10MB link.
I am getting about 20-30Kb/s ...
- 792
24
votes
6
answers
1k
views
How do I prove two files are the same legally?
We had someone steal some files before quitting and it has eventually come down to a lawsuit. I've now been provided with a cd of files and I have to "prove" that they are our files by matching them ...
- 2,455
24
votes
4
answers
5k
views
Pushing Large Files to 500+ Computers
I work with a team to manage 500-600 rented Windows 7 computers for an annual conference. We have a large amount of data that needs to be synced to these computers, up to 1 TiB. The computers are ...
- 340
23
votes
6
answers
51k
views
Recursively copying hidden files - Linux
Is there an easy way to recursively copy all hidden files in a directory to another directory? I would like to back up just all the settings files in a home directory, not the normal files. I tried:
...
- 459
22
votes
8
answers
23k
views
Doing an rm -rf on a massive directory tree takes hours
We're using rsnapshot for backups. It keeps lots of snapshots of the backuped up file, but it does delete old ones. This is good. However it's taking about 7 hours to do a rm -rf on a massive ...
- 32.3k
22
votes
1
answer
59k
views
Optimizing rsync for transferring huge files quickly
This is what I need:
Rsync daemon running on source server. Destination server does a rsync pull as below:
rsync -ptv rsync://source_ip:document/source_path/*.abc destination_path/
Huge binary files ...
- 405
22
votes
4
answers
21k
views
Rsync show progress for individual file
I'm using rsync to upload/download a pretty large file (20Gb). I'm aware of --progress option, but what it does in my case is just showing me the result at the end.
My problem is that during some ...
- 321
22
votes
2
answers
62k
views
What is stored in %Windir%\System32\LogFiles\WMI\RtBackup?
I occasionally notice in Resource Monitor hard disk activity related to ETL files in the folder C:\Windows\System32\LogFiles\WMI\RtBackup.
Which process/service creates these ETL files and what is ...
- 2,121
21
votes
9
answers
36k
views
Get total files size from a file containing a file list
I have a file containing a list of files that I would like to know the total files size. Is there a command to do so?
My OS is a very basic linux (Qnap TS-410).
EDIT:
A few lines from the file:
/...
- 454
20
votes
7
answers
4k
views
Linux mv file with long name
In Linux I sometimes rename files like this:
mv dir1/dir2/dir3/file.txt dir1/dir2/dir3/file.txt.old
Note that I want to just rename the file, not move it to another directory.
Is there a command ...
- 445
19
votes
9
answers
32k
views
Is it possible to hide lost+found?
Given a partition intended solely for storing music, video and so-on, is it possible to hide the lost+found directory?
- 14.5k
19
votes
1
answer
21k
views
What is fastest way to copy a sparse file? What method results in the smallest file?
BACKGROUND: I'm copying a sparse qcow2 VM image that is 200GB in size, but has 16GB of allocated blocks. I've tried various methods to copy this sparse file within the same server and have some ...
- 453
18
votes
2
answers
53k
views
nginx uLimit 'worker_connections exceed open file resource limit: 1024'
I keep getting this error in nginx/error.log and its driving me nuts:
8096 worker_connections exceed open file resource limit: 1024
I've tried everything I can think of and cant figure out what is ...
- 1,471
18
votes
3
answers
9k
views
Building a 1,000M row MySQL table
This question is reposted from Stack Overflow based on a suggestion in the comments, apologies for the duplication.
Questions
Question 1: as the size of the database table gets larger how can I tune ...
- 283
16
votes
7
answers
13k
views
In *nix, how to determine which filesystem a particular file is on?
In a generic, modern unix environment (say, GNU/Linux, GNU/Solaris, or Mac OS X), is there a good way to determine which mountpoint and filesystem-type a particular absolute file path is on?
I ...
- 715
16
votes
3
answers
51k
views
Is there a hard limit to the number of files a directory can have? [duplicate]
Wondering if there is a limit to the number of files that can be stored inside a directory, in CentOS 6. There is one particular directory which could potentially have millions of subdirectories.
...
- 677
16
votes
6
answers
9k
views
Files disappearing on linux server
I've got 4 specific files that seem to keep disappearing from a user's home directory. As far as we know, there are no cronjobs or other automated tasks that would be removing them. I've setup auditd ...
- 1,540
16
votes
4
answers
3k
views
Concatenating files to a virtual file on Linux
On a Linux system, is there any way to concatenate a series of files into one exposed file for reading and writing while not actually taking up another N bytes of disk space? I was hoping for ...
- 273
15
votes
6
answers
48k
views
Moving a Logical Volume directly from one server to another over the network?
I have a KVM host machine with several VMs on it. Each VM uses a Logical Volume on the host. I need to copy the LVs to another host machine.
Normally, I would use something like:
dd if=/the/logical-...
- 4,656
15
votes
7
answers
68k
views
How to Find a file in FreeBSD
I'm trying to locate all copies of example.filename on my FreeBSD server. What's the best / easiest / most efficient way to do this?
- 1,518
15
votes
4
answers
8k
views
read specified range of lines from a file
I have i file that contains 100000 line
how i can get the lines from line# 5555 to line# 7777 under linux.
Thanks for all.
ibrahim
14
votes
19
answers
28k
views
What is the simplest and fastest way to transfer large file through a Windows network?
I have a Window Server 2000 machine running MS SQL Server that stores over 20GB of data. The database is backed-up every day to the second harddrive. I want to transfer those backup files to another ...
- 417
14
votes
3
answers
31k
views
Systemd drop-in fails to create PID file
I have a drop-in for systemd-machined at the path /etc/systemd/system/systemd-machined.service.d/10-machined-pid-file.conf. when I run systemctl status systemd-machined I do see the lines
Drop-In: /...
14
votes
4
answers
27k
views
Monitor open process files on Linux in real-time
The files opened by a process can be found with this command:
ls -l /proc/PID/fd
Is there any way that can be done in a more interactive way like tail, auto-refreshing every x seconds?
- 449
14
votes
7
answers
26k
views
How long do you keep log files?
I have an application which writes its log files in a special folder. Now I'd like to add a functionality to delete these logs after a defined period of time automatically. But how long should I keep ...
- 243
14
votes
6
answers
36k
views
Best method to track folder size growth over time?
I have a file server that has a pretty large folder tree. There's a shared folder, under that are 5 departmental folders. Nested inside of those are thousands of subfolders and files.
I would like ...
- 463
13
votes
6
answers
56k
views
How do I delete a locked file in Windows?
I'm moving some files around on the server (Windows Server 2003), and there's a locked file that no one seems to have open. How do I delete this file?
user640