Mid-to-Senior Level OT/ICS SCADA Electrical Engineer
Job Title: Mid-to-Senior Level OT/ICS Electrical Engineer – SCADA Lab (Virtualization & Automation Lead)
Department: OT/ICS SCADA Lab
Location: San Antonio, TX
About the OT/ICS SCADA Lab:
The OT/ICS SCADA Lab is a dedicated facility focused on research, development, and testing of secure and resilient Operational Technology (OT) and Industrial Control Systems (ICS). We are committed to advancing the state-of-the-art in OT/ICS cybersecurity and virtualization. The Lab provides a dynamic environment for engineers to experiment with cutting-edge technologies, contribute to innovative solutions, and enhance the security posture of critical infrastructure.
Job Summary:
The Mid-to-Senior Level OT/ICS Electrical Engineer will be a technical leader within our OT/ICS SCADA Lab, playing a critical role in the design, development, deployment, and maintenance of complex, virtualized OT/ICS training and testing environments. This role demands a highly skilled and experienced engineer with deep expertise in virtualization technologies, infrastructure-as-code (Terraform and Ansible), network design, programming, and a comprehensive understanding of industrial control systems and cybersecurity principles. The ideal candidate will be able to work independently, lead projects, mentor junior engineers, contribute to strategic planning, and drive innovation within the lab. This position requires an engineer who can quickly contribute their expertise, leveraging a broad range of skills to meet the immediate needs of our projects while also shaping the future direction of our virtualization efforts.
Responsibilities:
· Virtualization & Network Design:
o Design and build complex, scalable, and secure virtualized OT/ICS environments using VMware ESXi.
o Perform OSINT/open-web research on network topologies within Levels 1-3 of the Purdue Model. Understand, explain, and implement these topologies, considering both technical feasibility and security implications.
o Create and present detailed network diagrams of devices based on requirements; obtain sign-off on network plans from customers and planners, ensure alignment with project goals and security best practices.
o Configure virtual machine networking according to established network diagrams, ensuring proper segmentation, security, and performance.
o Package and export VMs as OVA files with ESXi hardware compatibility.
· Infrastructure-as-Code (IaC) & Automation:
o Develop and maintain Terraform scripts for provisioning and managing virtualized OT/ICS infrastructure, ensuring infrastructure is consistently defined, reproducible, and scalable.
o Create and utilize Ansible playbooks for configuration management and automated deployment of software and services within virtual machines, ensuring consistent configurations across environments and minimizing manual intervention.
o Implement automation workflows to streamline the creation, deployment, and maintenance of virtualized environments, identifying opportunities for further automation and efficiency improvements.
· ICS System Integration:
o Integrate diverse virtualized industrial control system components, including sensors, actuators, PLCs, HMIs, and communication networks, ensuring seamless data exchange and control.
o Configure communication protocols (e.g., Modbus, DNP3, OPC) to enable interoperability between virtualized components, troubleshooting integration issues and optimizing communication performance.
· Open Source Research & Customization:
o Research and evaluate available open-source projects suitable for emulating OT/ICS hardware devices (PLCs, HMIs, Gateways, Connectors).
o Customize open-source code to create virtual devices and implement server-client relationships, enabling realistic simulation of ICS
environments and contributing back to the open-source community where appropriate.
· Programming & Configuration:
o Proficient in Python development, developing and modifying software programs and scripts to configure control systems, implement control logic, acquire data, generate alarms, and create visualizations.
o Configure communication protocols and network settings to establish connections between control system components.
· System Maintenance & Troubleshooting:
o Perform regular system maintenance, including applying security patches and monitoring system performance.
o Diagnose and resolve complex issues related to virtualized environments, network connectivity, OT/ICS component integration, and cybersecurity.
o Develop and implement proactive monitoring solutions to identify and prevent potential issues.
· Cybersecurity:
o Assess system vulnerabilities in virtualized OT/ICS environments, using both automated tools and manual techniques.
o Implement security controls to mitigate risks and protect against cyber threats, adhering to industry best practices and security standards.
o Monitor for potential security breaches and respond to security incidents, ensuring the integrity and availability of industrial processes.
o Contribute to the development and implementation of cybersecurity policies and procedures within the lab.
· System Documentation & Training:
o Document system configurations, procedures, and troubleshooting guides.
o Create technical documentation and user manuals to facilitate system maintenance and support.
o Provide training to operators and required personnel on the proper use and maintenance of control systems.
o Mentor junior engineers and share knowledge within the team.
· Compliance & Standards:
o Ensure that virtualized OT/ICS environments comply with relevant industry standards, regulations, and safety guidelines.
o Stay updated with industry best practices and emerging technologies to ensure the control systems meet the latest requirements.
o Participate in industry forums and contribute to the development of standards and best practices.
Required Skills & Experience:
· Bachelor of Science degree in Electrical Engineering, Computer Engineering, or a related technical field.
· Minimum of 5-7 years of progressive experience working with OT/ICS systems, with a strong focus on virtualization and automation.
· Proven track record of successfully designing, implementing, and managing complex virtualized OT/ICS environments.
· Current, hands-on work experience building ICS virtualized training ranges.
· Expert-level proficiency in using ESXi virtualized software and VMWARE to build and configure Virtual Machines (VMs).
· Expert-level proficiency with Terraform and Ansible for infrastructure automation and configuration management.
· Solid understanding of networking principles, including TCP/IP, subnetting, routing, and network security.
· Experience with industrial communication protocols (e.g., Modbus, DNP3, OPC).
· Proficiency in Python programming.
· Strong system maintenance and troubleshooting skills, with experience in diagnosing and resolving complex technical issues.
· Familiarity with cybersecurity principles and best practices for OT/ICS environments.
· Ability to work independently and as part of a team in a fast-paced environment.
· Excellent communication and interpersonal skills, with the ability to effectively communicate technical concepts to both technical and non-technical audiences.
· Demonstrated leadership abilities, with experience mentoring junior engineers and leading technical projects.
· Must be able to quickly contribute their expertise and provide technical leadership within the lab. This includes immediate proficiency with Terraform and Ansible in a virtualization context, as well as a deep understanding of OT/ICS principles and security best practices.
Security Requirements:
· Must be a United Stated Citizen and eligible to acquire TS/SCI clearance.
Work Environment:
· On-site at customer location in San Antonio, TX – 5 days per week.