Tag Info

Hot answers tagged

13

Because MySQL is so widely supported and this is really quite a trivial thing to do I would suggest going with it. Unless the server has at least a few GB of memory I would suggest sticking with MySQL rather than using an in-memory system. Once you start putting your data into a database, whether it's MySQL or something else, you will quite likely find that ...


12

Yeah, this was a nuisance I had to deal with a while ago. Here's a snippet from a script that dealt with converting a bunch of CSV files into mysql tables: COLLIST=`head -1 $table.csv` mysql --execute="LOAD DATA INFILE '$WORKDIR/$table.csv' INTO TABLE $table FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' IGNORE 1 LINES ($COLLIST); SHOW WARNINGS" > ...


11

You can use dedicated Key-Value type of NoSQL database which is optimized for this kind of tasks. Have a look at: Redis -- Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. MemcacheDB -- MemcacheDB is a distributed key-value storage ...


9

Why would you want to do that? Since you are not interested in adding or modifying data/metadata, you could just leave it as csv. CSV is associated with MS Excel by default, so whoever is going to open it, is going to get it opened in MS Excel. If that does not work for you for some reason, keep in mind, that, as far as I know, you can generate a (sort of) ...


8

Execute your sql query in the SQL tab of phpMyAdmin. After execution, scroll down the page and look for “Query results operations” Click “Export” link from the above and you will get the page to export all the results of the queries to desired format. That’s it.


4

You could use the Berkeley Database which does exactly this sort of thing, even if it hasn't been hip since the dawn of Perl5. Berkeley only supports key value pairs, and you tie the whole db to a hash and access it as such. Using Berkeley is well detailed in many of the older Perl references sitting on your shelf or try the Perldoc for the BerkeleyDB CPAN ...


4

You've flagged your question as amazon S3. I'd like to draw your attention to one of their other related products called Amazon SimpleDB. It sounds like the SimpleDB data model would fit well with your type of application. This is not a plug for it, but worth looking at especially if you're planning on using the Amazon cloud services. The SDB data model ...


4

And now for something completely different: Given: 180MB/3M products = 62 bytes/product on average. 30,000 queries per day = 0.34 queries per second Updated quarterly = essentially static data Outside of the box solution: Dump each product as a TXT resource record and store it in the DNS, e.g.: $origin products.example.com. product_1_name IN TXT ...


4

Easiest is probably to just count the commas: sed 's/[^,]//g' yourfile.csv | wc -c Normally you'd add one to get the number of elements, but if there's a newline there it's counted too. Convenient in this case, I guess. Also with awk: awk -F, '{print NF}' yourfile.csv


4

MySQL with MyISAM and some good indexes sounds perfect for this. There are a lot of other options of course, but MySQL is very widely (if not universally) supported on any commercial web host. Depending on the speed you require, memcached might also be worth looking at, but without knowing the size of each key/value pair, storing 3 million of them in memory ...


4

See Options > Query Results > SQL Server > Results to Text Check "include column headers". My text results are being enclosed in doublequotes so that should take care of the comma issue. Take a look at the "Results to grid" tab as well - I see a "Quote strings containing list separators when saving .csv results" If not through SSMS then you can also use ...


4

Did you try googling? I searched "rrd2csv" (thinking that it'd probably be named thusly, if it existed), and found: https://code.google.com/p/rrd2csv/ Alternatively, there's https://github.com/mscoutermarsh/RRD-to-CSV and in Python => https://gist.github.com/afternoon/947301 As it's Perl, and Open Source, you can customise it to meet your exact ...


4

If $UserName is the username, then: $Hdrive = "\data\Staff Homedrives\$UserName" Should do it.


3

I opted to try this out on my MySQL server to see what the problem was. It appears that you need to create the table in MySQL (with engine=csv specified) so that the table will appropriately be added to MySQL and it will keep track of the table field metadata. In my testing, I was unable to make MySQL learn of the csv file without first using "create ...


3

Was directed to the Try/Catch that does the error handling I need this fixes the problem. Try { [System.Net.DNS]::GetHostbyAddress($IPADDR) | Select Hostname,@{label='IP';expression={$IPADDR}} } Catch { Add-Content -Value "$IPADDR failed lookup" -Path ...


3

You're almost right, try this: id name address.0.line1 address.0.line2 address.0.city address.0.country address.0.postcode I inserted your sample document into collection bar in database test and then ran the export like this: ./mongoexport --port 31000 -d test -c bar -fieldFile fields.txt --csv > out.csv Then checked the results, which look good to ...


3

Use psloglist from http://technet.microsoft.com/en-us/sysinternals/bb897544.aspx It does exacty what you are asking for.


3

Like the others said the real problem is that for SHOW WARNINGS to work you have to be in the same connected session. mysqlimport disconnects from the session as soon as it's finished importing, making the capture of it's warnings impossible in it's current version. Sounds like an opportunity to donate a patch. :)


3

Uh, that sounds a lot like Microsoft Excel...


3

Logon Name is not one of the parameters that New-ADUser takes. See http://technet.microsoft.com/en-us/library/ee617253.aspx for a list of what you can use. Password won't work this way, as it has to be a SecureString for security reasons. If you want to load the password from the cSV you'll have to turn this into a fuller script that can convery the value ...


3

Take a look onto sc or Gnu Oleo, the may fit your need's Also a look onto the Emacs based Spreadsheets may be worth a look cheerio Steve


3

Instead of the export button at the top of the GUI, use the one at the bottom, within the "Query results operations" box. That is the one you want.


2

If you're looking at ways to backup Active Directory, the best and most reliable way to do that without creating domain controllers in some remote network is to use the VSS Writer for Active Directory with a VSS-aware backup utility. There is one that comes with Server 2008, and most off-the-shelf backup frameworks for Windows can handle that (possibly with ...


2

Sounds like you could use vim with ts/sw and friends to set up your columns widths, and then :1split to keep headers in view. You can also (assuming your CSV data doesn't have tabs in it already) do something like: %s/,/\t/g to switch commas to tabs, and %s/\t/,/g to switch them back, since it's easier to format TSV than CSV.


2

I don't know if there are limits for particular DB engines but I've met with some Department of Defense admins who manage a MySQL system which imports 10TB every day and they are not experiencing this kind of problem. Rather than use macros I recommend writing a script to parse the input file and create insert statements from it, which can be fed straight ...


2

No, you can't in the current form There are several options using UDFs or XML or dynamic SQL to pass in lists. The most comprehensive article on how to do this is "Arrays and Lists in SQL Server 2005 and Beyond" by Erland Sommarskog


2

You want the "-noClobber" option to ExportCSV it appears. http://ss64.com/ps/export-csv.html I also did an alternative solution that might work: # Get the list of computer names from AD $objDomain = New-Object System.DirectoryServices.DirectoryEntry $objSearcher = New-Object System.DirectoryServices.DirectorySearcher $objSearcher.SearchRoot = $objDomain ...


2

There is a python based solution on Sourceforge called csv2xls which may fit the bill. It does not appear to be maintained at the moment (last activity was over a year ago) but if it does what you need then that shouldn't matter too much.


2

Try using the ssconvert tool from "gnumeric" package. On Debian, install the package with sudo apt-get install gnumeric and then from the command-line, run: ssconvert file.csv file.xls This should do the job.


2

What does the directory really look like? Guessing the dir does not have dirs named 12,10,13... It does help us help you when we know what you're working with. Why not just use "basename"? PS C:\temp\post> ls Directory: C:\temp\post Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- ...



Only top voted, non community-wiki answers of a minimum length are eligible