Questions tagged [batch-processing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
55 views

Good technology for a large-scale batch operation on many S3 files in AWS Batch with Spot instances

I have an enormous corpus of text data stored in millions of files on S3. It's very common that I want to perform some operation on every one of those files, which uses only that file and creates a ...
Zorgoth's user avatar
  • 99
0 votes
0 answers
157 views

How to write a batch job query that has wait times, dependencies and conditional execution?

I'm trying to write a batch job that executes via task scheduler. The objective is to run these exe jobs sequentially. However, some of these jobs have dependencies on others. for eg. job1.exe (no ...
i_wonder's user avatar
0 votes
1 answer
3k views

Get the previous day date as variable in batch file from powershell command output

The scenario is like this. I need a previous day date (i.e. Today -1) as variable in a batch file. But if previous day is Sunday (i.e. script running on Monday) it should return the Saturday's date (i....
Niraj Jaradi's user avatar
1 vote
1 answer
727 views

Having issues with mkdir, robocopy, to date\time folders in a looping batch file

I have searched everywhere but cannot find a solution. Im unsure what is going on here but I desperately need some help in regards to this issue, Im running a batch file (Script for a game Server ...
Boss-Prime's user avatar
0 votes
1 answer
424 views

Several short consecutive SSH connections

Imagine performing a short repetitive process, which ends by doing a short lived SSH connection to some server (e.g. transferring a newly generated file via SFTP). This means that there will be a lot ...
user2340939's user avatar
1 vote
1 answer
3k views

Windows batch file scripting: how to get directory named with latest date (yyyy-mm-dd format)?

I have a directory with many subdirectories. Each subdirectory is named with the yyyy-mm-dd format. I'm writing a Windows batch file and I need to grab the directory name with the most recent date and ...
slantalpha's user avatar
0 votes
1 answer
4k views

Server load handling : One single batch request vs multiple requests

I was talking to a developer about different apis for create, update and delete and he was saying that it was better to have a single batch request to server to do operations like create, update and ...
Syed Suhail Ahmed's user avatar
1 vote
1 answer
2k views

How do I prevent additional jobs from a given user from starting?

With the Slurm workload manager how can I prevent more jobs from user bob from starting? Existing jobs should continue to run. The user should be able to submit more jobs but they shouldn't be able to ...
Levi Morrison's user avatar
-1 votes
2 answers
270 views

How to create a batch file which after some time execute a file but if interrupted execute another?

I want to automate my ms office installation 2007 or 2010 running a batch file. I am thinking having an options menu 1.Office 2007 2.Office2010 When started I want my timer to start..lets say 20secs ...
Maria Georgali's user avatar
0 votes
2 answers
9k views

Running supervisord process as root

My goal is to run sudo inside one of the programs: [program:doStaff] command=sudo python manage.py doStaff autostart=true autorestart=true stderr_logfile=/var/log/doStaff.err.log stdout_logfile=/var/...
guystart's user avatar
1 vote
2 answers
345 views

Containers for HPC batch processing

We are facing the problem that a lot of people want to run different scientific software on our high performance computing cluster. Every user requires a different set of libraries and library ...
J. Doe's user avatar
  • 13
1 vote
3 answers
7k views

windows start nginx and php-fpm in background in the same bat

I'm trying to create a bat file on windows 10 to start the nginx and php-fpm in background. At the moment I can do that but the command window (black window) remains open. If I close the window the ...
Doua Beri's user avatar
  • 113
-1 votes
1 answer
35 views

Scheduled tasks

Overview:- I need to download a csv file every 10 mins from an intranet site. This is done using a batch file. To ensure that it happens with out any delay, i have set up multiple tasks using Windows ...
Santhosh Rao's user avatar
1 vote
2 answers
1k views

How to tell if a DOS batch script is being run by task scheduler?

How can a DOS Batch command file determine if it is being run by Task Scheduler?
Dan's user avatar
  • 51
0 votes
0 answers
61 views

Enterprise batch processing of application-generated emails

Here is what I perceive is our problem. In our corporate environment, we have developed over 40 custom .Net applications (and growing). A good many of these send email notifications in some fashion. ...
ironfist's user avatar
  • 101
1 vote
1 answer
2k views

How to catch "The system cannot find the file" within For /F loop?

How to catch "The system cannot find the file" within For /F loop? I would like my code to do the following: 1 - Search for the specified file 2 - If file not found, handle the "The system cannot ...
Mr. Elliott's user avatar
34 votes
2 answers
4k views

Why is my bare-metal 16x 2.93GHz cores computer performing poorer than a VPS with 4x 2.5GHz cores?

I have a written a piece of multi-threaded software that does a bunch of simulations a day. This is a very CPU-intensive task, and I have been running this program on cloud services, usually on ...
Glauco Cattalini Lins's user avatar
4 votes
2 answers
9k views

Version v4.0.30319 of the .NET Framework is not installed and it is required to run version 3 of Windows PowerShell

I have a server which we are using for Batch processing. I can login to the server with svc_account and run powershell -f file.ps1 and it runs fine in either version 2 or 3 by the following - ...
whoisearth's user avatar
46 votes
11 answers
82k views

Most efficient way to batch delete S3 Files

I'd like to be able to batch delete thousands or tens of thousands of files at a time on S3. Each file would be anywhere from 1MB to 50MB. Naturally, I don't want the user (or my server) to be waiting ...
SudoKill's user avatar
  • 563
1 vote
1 answer
149 views

Will a Scheduled Task On Windows Server 2008 R2 work correctly if windows are opened and closed?

I have a batch script that I am currently running on my own Windows 7 computer as a scheduled task. The task runs nightly when my computer is locked. The task launches an Progress OpenEdge session ...
Bill's user avatar
  • 13
-1 votes
1 answer
460 views

Configuring / Securing credentials required by batch job

I have a batch job which should run as cron job on a Linux server. This job requires some credentials to do it's work. Idealy I don't want to have that kind of credentials stored on my server, but ...
Achim's user avatar
  • 149
1 vote
1 answer
2k views

Batch password change of all email in a cPanel account? [closed]

Does anyone know of a way to reset the password of one or more email addresses in a cPanel account from the command line?
Amged Rustom's user avatar
1 vote
0 answers
29 views

My transactional overnight batch has spiked in duration

I have an overnight batch run which is transactional...picking up data doing some small calculations and putting it somewhere else and it was finishing around 3am every day for the last 2 months. Now ...
Cheetah's user avatar
  • 241
2 votes
3 answers
15k views

Batch reset all iLO passwords via command line

I would like to batch reset all the iLO passwords on our HP Blade chassis without having to login to each one via the HTTPS website. How can I reset the root and administrator iLO passwords via the ...
Greg Bray's user avatar
  • 5,650
0 votes
1 answer
8k views

batch script to copy files as within SMB\CIFS protocol

I'm trying to copy files from remote machine using construction like this copy smb://DOMAIN;USERLOGIN:[email protected]/folder/ C:/tmp/ it shows next message - The filename, directory name, or ...
Olegdelone's user avatar
1 vote
2 answers
1k views

Low priority batch processes on live Mongo database

We want to run MapReduces on our live Mongo database, mainly so that we can extract metrics. However, we've had some bad outages caused by these MRs bogging down the Mongo server (in particular 100% ...
moinudin's user avatar
  • 205
2 votes
4 answers
8k views

How can I hide ms-dos window when running a .bat file?

I am running a .bat file for my script (Scheduled Tak (CronJob)) per minute. When it run, seems ms-dos window short period of time. My batch code like this; @ECHO OFF C:\wamp\bin\php\php5.4.3\php.exe ...
prestack's user avatar
-1 votes
1 answer
1k views

How to reliably run a batch job every 5 seconds?

I'm building an application where the sending of all notifications (email, SMS, fax) will be asynchronous. The application will write the notifications to the database, and a batch job will read these ...
BenMorel's user avatar
  • 4,607
1 vote
3 answers
298 views

Is there a way to see the path of a program executed from the Windows CLI?

If I type a command in the prompt, the command processor searches it in current path, and then (if not found) in the paths listed in PATH environment variable. So is there a way to easily know in ...
moudrick's user avatar
  • 111
0 votes
1 answer
341 views

How to delay and queue commands for execution at a later time? [closed]

My site receives numerous uploads and I can't afford to waste cycles and time processing them on the spot (e.g., trimming, optimizing, resizing, etc.). What technologies exist to delay and queue these ...
MarkL's user avatar
  • 13
-2 votes
1 answer
103 views

Does anybody wish to help a poor researcher publish a paper? [closed]

I don't know if this is a good place to beg for help, but here it goes: basically, I need to run a secure recommender system simulation (C++ console application) in order to meet tonight's deadline, ...
Mihai Todor's user avatar
0 votes
2 answers
1k views

Moving files with batch files from one pc to a server, to a another pc - worried about disk corruption

I use scheduled tasks that calls a batch file, that calls more batch files to move about three files from a pc, to a server, then to multiple other pcs. It all happens very quickly, as they are small ...
beauk's user avatar
  • 43
0 votes
1 answer
505 views

Distribute computing jobs across different machines?

I'm looking for a way to distribute computing jobs (mainly rasterization work) across multiple, idle machines on the network. The servers are running Debian 6 right now. Most of the work that needs ...
BastiBen's user avatar
  • 131
0 votes
1 answer
47 views

Batch Updates to Numerous Server 2008 Machines?

We run a lot of machines that are hosted in various datacenters (takes out the LAN based options) and we are finding the need to change Administrator account passwords for RDP (we use the same ...
Aidan Knight's user avatar
0 votes
2 answers
4k views

How to schedule batch file to run on the desktop in win 2008 R2

I have some backup batch file scripts which I was running in xp and I have migrated to windoes server 2008 R2 where it's not running in front when scheduled with task manager. I struggled a lot to ...
user53864's user avatar
  • 1,753
42 votes
6 answers
41k views

prevent xargs from quitting on error

According to the man page, xargs will quit if one of the execution lines exits with an error of 255: If any invocation of the command exits with a status of 255, xargs will stop immediately without ...
JDS's user avatar
  • 2,608
0 votes
1 answer
89 views

Whats the best way to execute high processing tasks on low RAM VPS

I have one VPS on internet and that have only 768MB of RAM. Now i have run crawlers and various Optimzation algorithms /Data Mining Algos on the one mysql database. I think i need better computer for ...
Mirage's user avatar
  • 599
3 votes
2 answers
5k views

Torque jobs does not enter "E" state (unless "qrun")

Jobs I add to the queue stays there in "Queued" state without attempts to be executed (unless I manually qrun them) /var/spool/torque/server_logs say just 04/11/2011 12:43:27;0100;PBS_Server;Job;16....
Vi.'s user avatar
  • 841
0 votes
1 answer
467 views

Only half of my pbs/Torque jobs are being scheduled

My supercomputing center recently moved from SGE to pbs/Torque. Now, when I schedule my array jobs, only half of the jobs in the array get scheduled. When they finish, the other half get scheduled. ...
vy32's user avatar
  • 2,118
2 votes
2 answers
337 views

Batch rename via command line?

I have a Media Temple (dv) server which I am setting Magento up on. I have uploaded a few thousand images to the /media/import directory that the client has provided me with. The problem is, some ...
dannymcc's user avatar
  • 2,727
0 votes
2 answers
149 views

Batch File Printing

I have a user that has to print out about 220 documents averaging 2-3 pages each. When printed using the standard Windows method of Select All - Right-click - Print it overwhelms either the print ...
chrispt's user avatar
  • 111
2 votes
2 answers
1k views

X11 for apache user

We are using inkscape to convert SVG images uploaded to our server via a web form. For this inkscape offers a batch mode via the -z option, but this batch mode has a flaw: When inkscape is run by the ...
fuenfundachtzig's user avatar
2 votes
1 answer
2k views

VirtualBox on Windows Hosts | Export and Import batch automate

VirtualBox on Windows I am planning to deploy virtual box on around 10 computers (Vista / 7), I have setup a centos vm and exported it as an appliance with the ovf and vmdk files in the same folder. ...
Mutahir's user avatar
  • 2,357
6 votes
9 answers
17k views

Remove files that are listed in a text file

I have a file that exported a bunch of file names that need to be removed. I need to know how to go about removing each file without having to issue it one at a time at the command line. I've ...
drewrockshard's user avatar
0 votes
3 answers
4k views

Zip and FTP a File via a batch script

I am trying to ZIP and FTP a file in one batch file. The Zip part works, but the FTP application reads the entire batch script, rather than just the lines below the FTP commands, causing errors If ...
mmcglynn's user avatar
  • 365
1 vote
1 answer
729 views

How to batch edit a list of files?

I have a list of files where I need to remove some lines that have been added yesterday by a spambot. The section I want to remove looks like this: ^M <script>[...] bunch of malware code [...]&...
Peter Kruithof's user avatar
3 votes
1 answer
319 views

Backup multiple Exchange Accounts without direct access to exchange server

For e-mail, we use Microsoft Exchange and it is hosted by 1and1.com. We have about 30 Exchange accounts that I would like to backup to a PST file. That is, for each account that we have (all 30), I ...
user avatar
0 votes
2 answers
378 views

Windows Job Scheduler

I'm looking for a Windows Server job/task scheduler for a small business. We need more functionality than the built in tools, including SQL Server Agent. We need conditional execution and would like ...
user avatar
3 votes
7 answers
226 views

Software for Utilizing Idle Workstations on Network

I work at a company that maintains a network of about 200 Windows XP workstations, and several of them are on (locked or logged out) with an idle processor and 2GB of RAM that is just sitting there. ...
Jason Coon's user avatar
3 votes
1 answer
8k views

Running 'top' with 'Irix mode' off in batch mode

Can anyone tell me how I can tweak the following top command so that it always runs in Solaris mode in a batch mode operation? top -d 3 -n 2 -b -p <PID> I need 'top' to output a process' CPU ...
His's user avatar
  • 249