Questions tagged [automated-testing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
5 votes
4 answers
221 views

Automated software testing on real hardware

I'm currently working on a small testing farm at work and especially the part of resetting the machines after a test was run gives me some headaches. Before we decided to get a couple of dedicated ...
4 votes
2 answers
186 views

Website diff testing tool [closed]

Our CEO wants us to check all of our 100+ hosted/partner websites. Not only does she want to check if they are up and running, but she would like a diff analysis on the websites so that we can keep a ...
1 vote
1 answer
834 views

Puppet, test roles in control-repo with beaker-rspec

I want to test my control repo (having profiles and roles in it) with rspec-puppet and beaker-rspec. I am still having some trouble with beaker-rspec: I would like to test the profiles individually, ...
0 votes
1 answer
94 views

Test cases to verify that VM image is working

I'm currently working on my homelab using automation. I'm currently building Ubuntu images which are supposed to be distributed to my students when they are having labs. I'm wondering: Is it possible ...
7 votes
2 answers
4k views

How to detect timeouts in an expect script

Consider an expect script that spawns a command and waits for a sequence of events. We use this to test software. I'm trying to always get a failure return value from the script when there is a ...
0 votes
0 answers
54 views

Configurable Stateful Traffic Generator Advice

I need a traffic generator tool which I should be able to configure followings; Number of Packets Number of Sessions Number of TCP, UDP packets Average number of packets in a session Average packet ...
0 votes
1 answer
937 views

Jenkins builds not marked unstable/failed if PHPUnit fails

So, the title pretty much gives it away. We're using Jenkins 2 with PHPUnit 6.5.12 and a couple of days before, everything worked fine. Now, every build we do and has failures (or worse, errors) in ...
7 votes
3 answers
2k views

Salt State Testing via Jenkins

All, We're trying to have automated tests in our Jenkins setup to run "smoke" and "lint" type of tests in our salt state files (.sls). All google-foo so far has yielded very little information. There ...
1 vote
3 answers
22k views

Running a command a different user using `su` gives "standard in must be a tty"

I am using STAF to automate db2 interface testing with our product. I am able to run the command using su - db2inst1 -c "/home/db2inst1/sqllib/bin/db2 list database directory" but when I try the same ...
1 vote
1 answer
961 views

serverspec if statement for mysql version on server

Is there a way to check which package installed on server by command output? describe command('mysql -e "select version();"') do if its(:stdout) { should =~ /5.6.27-76.0-log/ } describe ...
18 votes
5 answers
9k views

Best practice for testing chef recipes?

My team has created a large number of chef cookbooks. I was wondering what methods and frameworks I should look into so we can start creating tests to assure that our nodes are configured properly?
0 votes
1 answer
88 views

Multi-stage CI monitoring or how can I split the deployment steps into controllable chunks

First of all, I am not looking for a product recommendation (well, I am, but it's more like me looking for a solution to a problem and not trying to pick something from an existing list) The ...
1 vote
1 answer
197 views

Automated Installation Testing on Windows [closed]

I am looking for some software I can work with that will allow me to test that programs were installed and are functioning correctly. Instead of opening each program manually and see it working, I ...
0 votes
2 answers
198 views

Auto test framework for server deployment

I deploy Linux servers all the time and would like to test each installation. Like a system test. For applications there a numerous of test frameworks, but I can't find anything for testing the a ...
3 votes
2 answers
6k views

Automated test suite for linux servers and network devices?

Coming from a development background i'm used to automated (unit) testing before stuff goes live.Now i'd like to use the same approach to (new) linux (and some windows) servers and networkdevices. I ...
2 votes
0 answers
415 views

Jenkins: Build all commits triggered by push

How can I make Jenkins build all new commits (not just the latest) when there is a push? Note: The push is the trigger for the Job
2 votes
2 answers
4k views

Trouble getting transparent Squid forwarding proxy working with ssl

I have an environment consisting of four servers networked together. One server acts as the server, and the other three act as clients for running automated tests and Linux benchmarking using ...
6 votes
1 answer
4k views

How to test for file absence with serverspec?

The serverspec guide on resource types doesn't explain how to test for the absence of a file, rather than its presence. This is the best I could come up with: describe command('/bin/bash -c "[[ ! -e /...
9 votes
1 answer
9k views

Automated hardware testing of HP servers?

As part of provisioning servers we run HP's Insight Diagnostics to test the hardware. This is a manual process. Is there a way automate the running of Insight Diagnostics? There is the hpdiags ...
0 votes
1 answer
44 views

Static Analyzer for Configuration files

I'm currently planning to analyze csv-reports from our ActiveDirectory and configuration files from our switches with Python. I'm thinking of having seperate modules for each type of data (csv, txt). ...
0 votes
1 answer
216 views

What URL should I use to test IBM cognos Mobile, and what are the expected results?

I need to test IBM Cognos Mobile functionality and need to have an accurate baseline/reference point. What URL should I use to test Cognos? Is it https://server.somthing.com/IBMCognos/M <-- ...
14 votes
2 answers
2k views

How can I write automated tests for iptables?

I am configuring a Linux router with iptables. I want to write acceptance tests for the configuration that assert things like: traffic from some guy on the internet is not forwarded, and TCP to port ...
3 votes
1 answer
4k views

How to automatically create a Hyper-V VM and install an app using PowerShell

I am designing a testing infrastructure for a WPF application. I want a system that will create a VM when a new MSI appears in a folder, and install the MSI on the newly created VM. I am using Hyper-V ...
1 vote
1 answer
293 views

Testing and workflow capabilities keep failing

My testing and workflows are failing. The workflow fails for this reason. But I have the TFS build service on the environment and the agent has the green light beside it. The testing fails for this ...
2 votes
1 answer
300 views

Continuous Backup Testing?

Is there any software/setup that allows for automatic backup testing daily? by testing i mean test deployment, check setup post deploy warn on bad result? would be nice to use some kind of VM to ...
1 vote
1 answer
2k views

Managed Ethernet switch per-port MAC address translation?

Just a shot in the dark here, but I thought I'd ask in case anyone has some ideas: I've got a testing scenario where some (GUI-less/embedded) IPv6 devices are going to be temporarily plugged into the ...
4 votes
2 answers
4k views

Too much memory consumed during TFS automated build

We're running TFS 2010 Standard Edition, and we've set up an automated build to run whenever someone checks in code. We run through all of the automated tests (built with MSTest) as part of the build. ...
3 votes
4 answers
133 views

Managing a Testing Infrastructure

We're currently running continuous automated functional tests against the CI builds of our software. This is done on a variety of Windows based machines (around 10-15 boxes or VMs), to identify ...
0 votes
1 answer
105 views

What could be a good configuration for a testing machine?

In Zentyal we use virtualization (ANSTE) to perform automatic tests that require scenarios with several machines interacting in one or several networks. Currently we have a multicore Xeon machine with ...
11 votes
5 answers
1k views

Test-driven development for infrastructure deployments?

I've been using puppet for deployment of infrastructure, and most of the work I do is with Web 2.0 companies who are heavily into test-driven development for their web application. Does anyone here ...
0 votes
3 answers
2k views

Diagnostic Live CD

I'm looking for an automated diagnostic live cd/usb drive for testing all hardware independent of the operating system. I'm also looking for an automated anti-virus/anti-spyware live cd/usb drive ...
0 votes
1 answer
173 views

Is there a scriptable method for testing duplex settings on network interfaces?

I'm looking to test, on a large number of servers, whether or not the duplex settings on the network interfaces have been set up properly. Is there a tool, or set of scripts, which I can run against ...
1 vote
1 answer
649 views

Saving multiple hibernate states on Windows

This question is related to this one: Automated windows restart testing In order to automate testing in Windows, it could be useful to save multiple hibernate states and running them to see whether a ...
1 vote
1 answer
8k views

How can I set up another alias like "localhost" to point to a folder within htdocs?

So I have xampp running, localhost or 127.0.0.1 will take me to htdocs within my xampp folder. Is there a way I can use a similar name such as localhost2 on the same machine to take me to folder, lets ...
1 vote
4 answers
156 views

Automated Image Testing

Let us say I'm using a packaged-based approach of imaging for the Mac, and have a freshly imaged machine (or better, several different types of machines) ready to test. How do I perform automated ...
3 votes
1 answer
133 views

Hardware/Network equivalent to Automated Testing? [closed]

I'd like to build out my network infrastructure by using a TDD approach. Specifically, I'm not sure what monitoring we will require, but if something fails, I'd like to be able to fix is then put in ...