541 reputation
37
bio website scottmcclenning.blogspot.com
location Oxnard, CA
age 41
visits member for 4 years, 8 months
seen Aug 16 at 17:19

Jan
26
comment Undelete ZFS file shared by Samba
I've heard of people editing the disk to have ZFS use a previous state, but I don't know of a utility to do that. This would allow you to get to the files if they hadn't been over written. In addition, I've heard of some data carving tools that would be used on ZFS, mainly for forensics.
Oct
24
comment Recommended technology for layered disk cache in linux
@SpliFF: Just in case you hadn't seen this presentation on BTRFS. I thought it was interesting, but I have been out of the BTRFS loop for a while so there may be nothing new to you in the presentation. event.on24.com/eventRegistration/…
Oct
23
comment Recommended technology for layered disk cache in linux
@SpliFF: I'll have to check out btrfs, and bcache myself. I'm hoping for an arms race between ZFS and BTRFS, like of like Intel and AMD. With competition both sides are forced to innovate and the winners in the end are the users.
Oct
20
comment Recommended technology for layered disk cache in linux
I hadn't used it, but it appears ZFS is being ported to Linux. zfsonlinux.org If ZFS isn't the answer, I believe it soon will be.
Oct
18
comment Recommended technology for layered disk cache in linux
I only have a little experience with ZFS. In case you didn't now ZFS love lots of RAM (or L2ARC). I saw a video that said the default cache size is 80% of RAM (if that RAM isn't needed by something else), but that is tunable. Dedupe uses RAM so if you don't need hard drive space, you may want that RAM be used for caching. I linked to all seven videos on my blog scottmcclenning.blogspot.com/2010/10/zfs-cache-and-tuning.html If you want to learn about tuning ZFS, the videos and his blog should help.
Oct
18
comment Generating error correction data for existing files (backups)
+1 Used PAR2 on my 7z backups I burnt to DVD because not all DVDs are 100%.
Oct
13
comment how-to: ZFS design and cache tuning : ARC, ZIL
If you want a video series where they talk about how ZFS works and some tuning goto youtube.com/user/deirdresun and it is called "ZFS in the Trenches: Ben Rockwood at LISA 09" (7 in all). Ben talked about turning the cache and how to monitor ZFS. His blog is at cuddletech.com/blog I'm not sure that bug is still active, it isn't posted as fixed, but a workaround is posted in a related bug. When you go to that bug it was fixed in snv_56. Not sure if the second bug fix, fixed the first bug. I don't remember a reference to memory leakage in the video series.
Oct
11
comment Tips on efficiently storing 25TB+ worth million files in filesystem
After thinking about it, you’re right; DFS may be able to be used if you have a DFS root point to the folders living on other machines. That way the user would see one file structure and wouldn't need to know which machines the data actually lives on, DFS would know. That would work. Usually when I have people ask me about Windows DFS, they usually think it is a way to pool together storage space, and that is why I just to that conclusion. Sorry and your right that could work.
Oct
11
comment Tips on efficiently storing 25TB+ worth million files in filesystem
FYI: Windows DFS is a way to keep files/folders on multiple server in sync. It won't allow you to use the storage on multiple servers as a single storage drive. microsoft.com/windowsserversystem/dfs/default.mspx
Oct
10
comment ZFS Sync over unreliable, slow WAN. ZFS replication, or rsync?
In my case, I'm still learning rsync so I may not have set things up properly or was using rsync improperly, but eliminating rsync from having to check files that didn't change or just doing a copy was faster for me. (I was trying to use rsync to mirror some data on an external drive.) Perhaps what killed rsync was several thousands of small files. (Small being a few KB each.) I would expect with fewer and larger video files rsync would perform better. In my case I had to test and test to see what actually performed best for my setup.
Oct
10
comment ZFS Sync over unreliable, slow WAN. ZFS replication, or rsync?
The only thing I would add is maybe not using rsync. I too experienced the slowness of rsync because I was using it as a transfer process, not sync process. Then I realized most of my existing data didn't change and only new data needed to be copied, for me, I used cp on only the new files and it was much faster. If I had files that changed (or only parts of files) then I would use rsync. So I suggest separating the new files out and pick a resumable transfer method. Also, compression would be a CPU & RAM/bandwidth trade-off (on both ends).
Jan
21
comment Samba vs Windows File Sharing/Server, which one is faster?
I know this could be a religious war and I hadn't seen any benchmarks or done any tests myself, but my anecdotal opinion is samba was faster. However, when I look for a central file store I look for ease of administration and backup/restore. That for me usually determines the OS I use, which for me means Windows.
Jan
21
comment Samba vs Windows File Sharing/Server, which one is faster?
I know this could be a religious war and I hadn't seen any benchmarks or done any tests myself, but my anecdotal opinion is samba was faster. However, when I look for a central file store I look for ease of administration and backup/restore. That for me usually determines the OS I use, which for me means Windows.
Jan
9
comment Is there something (free) like ZFS or HadoopFS or GFS for Windows?
Agree with Matt, DFS is a way to replicate a directory across multiple servers. Not a way to utilize multiple server's storage as one directory.