43
votes
How to fetch multiple files from remote machine to local with Ansible
You should use the synchronise module to do this. This uses the awesome power of rsync. It will copy file & directory structures of any depth, is bulletproof and highly efficient - only copying ...
- 1,792
35
votes
How to copy file preserving directory path in Linux?
You can also use rsync -R, which works on OSX where cp --parents isn't available.
https://stackoverflow.com/a/13855290/598940
- 449
32
votes
Accepted
How to fetch multiple files from remote machine to local with Ansible
You will probably need to register remote content and than loop over it, something like this should work:
- shell: (cd /remote; find . -maxdepth 1 -type f) | cut -d'/' -f2
register: files_to_copy
-...
- 535
19
votes
Accepted
What's the name of this windows feature?
Sounds like UAC Virtualization.
Any writes to a system folder (or Registry) as a standard user were redirected to a location in the AppData folder so those apps that incorrectly assumed that all ...
- 656
13
votes
How to let 'cp' command don't fire an error when source file does not exist?
Old question, but might still be relevant for others.
If you don't need to use cp, you could try with rsync.
To copy all files from a source to a destination directory, run:
rsync -avzh --ignore-...
- 311
13
votes
How to let 'cp' command don't fire an error when source file does not exist?
Piping the result to true ensures that the command will always succeed. I have tried this on Linux but not on any Mac OS:
cp ./src/*/*.h ./aaa | true
- 231
13
votes
Accepted
Cannot cancel copy task in ESXi Web interface
On Esxi only, not ESX: Restart the VM client daemon after SSH Login
/etc/init.d/hostd restart
- 255
10
votes
Accepted
What is fastest way to copy a sparse file? What method results in the smallest file?
From the above benchmarking, it looks like using dd on our target hardware with a blocksize of 64K gives us the best overall result considering the copy time and bloat:
dd if=srcFile of=dstFile iflag=...
- 403
7
votes
any issues with file-copy Git Repo to another machine (and keep working from both)?
There is nothing wrong with repository move from one machine to another.
Git will update its internal configurations (if needed) on first run into the repo.
Maybe you can ask yourself why you need ...
- 171
7
votes
Accepted
Copy (sync) timestamps only of already copied files and folders in Windows
Ok, I've managed to do this maybe it will be useful for someone else:
robocopy /e /timfix /dcopy:t c:\source d:\destination
Does the job :)
- 241
7
votes
How can I get docker build to overwrite a file?
I experienced this too, but after much testing discovered that I had a typo in one of my COPY directives' destination paths.
Double-check where those files are really going
-COPY nginx.conf /etc/...
- 215
7
votes
What's the name of this windows feature?
You are probably referring to the sandboxing done on processes with "low/medium integrity" flag. From Wikipedia:
Windows Vista and later editions include a "low" mode process running,
known as "...
- 45.5k
6
votes
Accepted
Copy directory contents with Ansible (without replace destination files)
by default ansible forces overwrites, maybe disabling it would help your case (force=no).
- 1,211
5
votes
How to fetch multiple files from remote machine to local with Ansible
i dont have enough rep to comment otherwise i would add it.
I used what Kęstutis posted. i had to make a slight modification
- shell: (cd /remote; find . -maxdepth 1 -type f) | cut -d'/' -f2
...
- 151
5
votes
Accepted
VMware NIC Teaming - How to gain more than 1Gbit/s copy speed between two VMs?
A NIC team does NOT provide any more bandwidth. A single network connection (like the one between your 2 VM's) will never use more than one of the possible NIC's in the team. The point of a NIC team ...
- 1,471
5
votes
Move files to another directory which are older than a date
Be careful when using the above solutions, I used them and ended up moving all files in all subfolders!!!!
This command moves all files in /source directory and all subfolders under source directory:
...
- 151
4
votes
What is the windows command line command to copy files?
Use ROBOCOPY if you're creating backup scripts. xcopy has been deprecated and will likely be phased out of use in the near future. robocopy can do everything xcopy can. It is also more flexible and ...
- 141
4
votes
How to copy file preserving directory path in Linux?
I use cpio in combination with find. Explanation here.
Example for your use case:
find /a/myProject/.project/ -type f | cpio -p -dumv /b/.
This command finds all files in /a/myProject/.project/ and ...
4
votes
Accepted
What is the most efficient way to transfer files from AWS S3 to S3?
https://aws.amazon.com/premiumsupport/knowledge-center/account-transfer-s3/
You can copy Amazon S3 objects from one AWS account to another by using the S3 COPY operation. You must give the ...
- 32.7k
4
votes
Accepted
Why robocopy with multi-treading copies faster
It's faster because you're copying more than one file at once.
It's possible to overdo it; if you use /MT:64 (for example) on a desktop with limited resources it'll grind to a halt, but in general: ...
- 18.6k
4
votes
Accepted
copy large number of files over ssh
…to answer the original question as stated…
There are two things to discuss here.
Using SSHFS
SSHFS uses the SFTP "subsystem" of the SSH protocol
to make a remote filesystem appear as if it were ...
- 1,120
4
votes
Accepted
Copy-paste heredocument into Bash converts tabs into dots
Bash is doing filename completion inside the heredoc. Your directory has no non-hidden files (files that don't start with '.'), so the longest substring of characters that match the beginning of all ...
- 388
4
votes
Rsync huge dataset of small files 5TB, +M small files
Try xargs+rsync:
find . -type f -print0 | xargs -J % -0 rsync -aP % user@host:some/dir/
You can control how many files to pass as source to each call of rsync with -n
E.g. to copy 200 files at ...
- 2,701
4
votes
Windows file copy - intermediary
If you're on workstation C and issue a copy command from server1 to server2, all the bits will travel from server1 to workstation C to server2. So yes, if you've got any significant amount of data, it ...
- 35.9k
4
votes
Copy (sync) timestamps only of already copied files and folders in Windows
Copying it again might be the fastest method. Attended oversight of all commands took me hours and it wasn't perfect.
To avoid file copy, added copy:t
robocopy /e /timfix /dcopy:t /copy:t c:\source d:...
- 66
4
votes
Accepted
How to merge two files into single file using ansible in remote server
This works. It reads the contents of all files and reduces the resulting array of lines to unique values. Then a new file with those lines is created.
- hosts: localhost
gather_facts: no
vars:
...
- 21.7k
3
votes
Accepted
Robocopy Overwite IDENTICAL files but Rename Similar Files
Nothing beats a readthrough of the Robocopy documentation.
To cover all your objectives using Robocopy I am fairly certain you would need to stage the files in some way first. Such as identifying ...
- 4,726
3
votes
How to fetch multiple files from remote machine to local with Ansible
Fixing the example above
- hosts: srv-test
tasks:
- find: paths="/var/tmp/collect" recurse=no patterns="*.tar"
register: files_to_copy
- fetch: src={{ item.path }} dest=/tmp
...
- 31
3
votes
Log backup clashes with log rotation
It doesn't look like logrotate provides anything that you can hook into (e.g. a pid file) to solve this problem so you'll need to work around it.
I would look into using flock(1) which is available ...
- 115k
3
votes
Rsync : copying over timestamps only
Use the source (e.g. /path/to/source) directory as reference for the touch command. Just cd to your target directory and do
find -type f -exec touch -r /path/to/source/{} {} \;
You cannot copy sub-...
- 31
Only top scored, non community-wiki answers of a minimum length are eligible