Skip to content
Factory Field Notes
← All episodes
EP_07Jun 19, 2026· 41 min

Industrial Automation Career Advice: How to Get Better, Get Hired, and Get Paid More [Q&A]

Industrial automation careers reward the people who treat skill building like a capital investment, not a hobby. This Q&A breaks down how to grow in controls, SCADA, and MES, how to break into the field without an engineering degree, and how to think about which skills actually pay.

Q&ACareerSkillsSCADAMES

Watch

Show notes

Industrial automation careers reward the people who treat skill building like a capital investment, not a hobby. This Q&A breaks down how to grow in controls, SCADA, and MES, how to break into the field without an engineering degree, and how to think about which skills actually pay.

Vladimir Romanov, founder of Joltek, answers real questions from the community as someone who has hired, mentored, and built automation teams. If you manage engineers or you are trying to grow into a more valuable role yourself, subscribe for breakdowns that connect the technical work to the business decisions behind it.

The throughline of this episode is simple and uncomfortable: interest alone does not set your market value. The people who pull ahead pair what they enjoy with what the industry will actually pay for. An engineer who can consult on the architecture of a SCADA and MES platform commands a premium over one who only produces AutoCAD or EPLAN drawings, not because one role matters more on a given day, but because the harder skill takes more years to master and carries more uncertainty and communication load. Treating your own learning like a portfolio, weighting it toward marketable and in demand skills, is the single most useful habit discussed here.

That same lens reframes a career change. One viewer asked about moving into automation in his 40s from law enforcement and a Salesforce administrator role, with no engineering degree. The honest answer is that the most direct path for someone fluent in business IT, CRMs, and ERPs is SCADA and MES, not the electrician route, because the work feels familiar and the transition story writes itself in an interview. Inductive University from Inductive Automation is free and will take you a long way on the SCADA side before you ever pay for formal training.

For managers, the rest of the episode is just as relevant. There is a detailed answer on when to deliberately limit a system's scope rather than over deliver, including how time and materials contracts shift risk to the customer while fixed bids shift it to the integrator, and why every extra mile should be a conversation with the customer before it becomes free work. The final question walks through a real OT network fault where two Rockwell devices ended up sharing an IP address behind NAT modules, and why DHCP in a production environment, aging battery backed controllers, and a single moved cable are the usual suspects.

Learn more at Joltek:

Timestamps

0:00 Heading to Automate 2026 and What This Q&A Covers

0:55 How Do You Actually Get Better at Your Controls Job

8:18 How to Learn PLC Programming on Your Own

13:50 Who Is Joltek

14:25 Career Change Into Automation Without an Engineering Degree

24:05 When to Limit Scope on Purpose: T&M vs Fixed Bid

33:45 OT Network Fault: Two Rockwell Devices, One IP Address

39:40 Closing and How to Reach Out

If you are weighing a skill investment, a hire, or a career move in automation, send a note on LinkedIn or leave a comment. Vladimir reads and responds.

Come find the team at Automate 2026 in Chicago.

Transcript

How's it going, everyone? Welcome to yet another video in which we're going to discuss some of the most upvoted comments and questions in our industry, which is industrial automation and manufacturing. I am currently preparing to head over to Automate twenty twenty-six, which takes place in Chicago this year.

And so if you're going to be there, make sure to reach out to me either by leaving me a comment in the YouTube channel or send me a note on LinkedIn. You can find me very easily there. I am highly responsive. Appreciate all the feedback, appreciate all the comments on all the videos. We're going to dive right into this.

I have released another video covering the technical side of what I'm presenting at Automate. So if you want to check that out, please do that being said, let's get into the currently popular questions and comments

So our first question of the day comes from the user by the name of EmperorTaiLong, and he says, "How do you get better at your job controls? As the title says, how did-- do you personally get better at your job? Is it just by years of experience, or do you go out of your way to learn more? And to those that learn more, how?

Reading manuals, scrolling, enrolling in classes, asking questions? Sometimes I just don't really want to jump into the rabbit hole of learning something too niche or learning something that doesn't benefit me as I would probably not use it." So this is a dilemma, I would say, for all of us, so in the world of industrial automation and manufacturing in a broader sense, there is just a vast amount of knowledge on a variety of different topics.

And it is very diff-difficult to pick if you're learning more on the electrical side, if you're learning more on the mechanical side, if you're learning more software, if you're learning tools like Lean Six Sigma that are more specific to manufacturing operations, there's always something to learn.

There's always a different certification that you can attain. There's al-always different goals to strive for. I would say that as I get older, this is, of course a challenge, but at the same time, in certain cases, a blessing, so in university while you're doing a very structured program, there is always the next milestone.

You always have courses, you always have semesters, you have your exams, quizzes, what have you, so that you can then get a final degree for that program. In life, in general, there is no structured path to get to where you need to go. It is really up to you to figure out what to prioritize. So in a very broad sense the best advice that I can give you is that a lot of times people will have their own hobbies and interests, but it is also important to keep in mind what brings in the money to support your outside of work type of goals, so ultimately, you are making an investment in yourself when you're trying to learn, but you want to pair the interests with opportunity. And what I mean by that typically is, and I'll give a couple of examples as we progress into this answer, is that maybe you enjoy programming more, so then you start to choose maybe instead of doing pure control systems where there's still a lot of electrical design, there's still a lot of troubleshooting, you naturally navigate towards SCADA and MES, where it is a lot more software-based.

If you enjoy more of the mechanical aspects, maybe you can start gravitating towards AutoCAD, SolidWorks. There's plenty of jobs doing panel design as well, so where the software elements are a lot more limited. Maybe you enjoy the people aspect, so then you start navigating towards project management, Lean Six Sigma,

there's a ton of different certificates on that side Maybe you're interested in the cybersecurity side. And for those of you who are maybe younger, or I would say earlier in their career, I think it's important to understand and experiment with all of those facets before diving in deep, and experimenting could mean a variety of different things.

It could be talking to experts that have spent many years in those fields. It could be taking a smaller, maybe entry-level certificate. It could be watching a bunch of different YouTube videos or Udemy tutorials and courses before committing to one aspect or the other, so let's get back to the question, reading manuals, enrolling in classes, asking questions.

And I don't think there's a one-size-fits-all answer, at least for me. So I still do a lot of work in the field. I like to download videos from both YouTube and Udemy, and then watch different experts explain some of the technical topics. I do purchase classes, again, on, like I said, Udemy and specialized platforms, but also sometimes from the accredited institutions,

so I will be spending the capital, for example, to get certified on Ignition gold, which then opens up other doors and other opportunities at the SCADA and MES layers, so that's what I'm investing in. I know many individuals that have been quite successful. I've mentioned this a little bit earlier.

On the project management side, they went out and they got their PMP certificate, and I believe there's one below that before you get the years of experience of project management that you can also attain, and then at that point, you need to pay usually the official certification program, and you have the choice of either taking the knowledge by reading the books.

You can purchase the books on Amazon or wherever. You can buy the courses on Udemy, or you can go to a specialized training for project management. So there is no right or wrong answer. There's going to be technologies that you can master on your own. So I've talked a little bit about AutoCAD. There's ePlan.

There's a few other solutions that can do electrical CAD drawings if that's what you're interested in. You can go and learn from them directly. They have their own sort of like training or pseudo university programs where you can step-by-step understand the solution with their trial, then implement, and you would have gotten the knowledge necessary to then be up to, I would say, like skill in your specific job.

And again, the question is how, i, hopefully we answered that. Then the second question is what to learn. And like I said, in our industry, but just in life in general, hopefully you are paying attention as to what is in most demand for your specific skill sets right now and where the industry is going,

so I can tell you firsthand that when you're in a position of an engineering manager, engineering director, or VP of engineering, you need to make the choices of who do we bring onto the team. And you look at the costs as well as the ROI for that person And to be completely blunt, someone who does AutoCAD work or someone who does EPLAN electrical work is going to be a lot cheaper than someone who can consult on the architecture of SCADA and MES solutions for the facility.

It is simply a fact of life that certain jobs will command a premium because it is more difficult to master. It takes a lot more years of experience. It takes perhaps a broader understanding that is not just the technology, but also how to manage people, how to deal with a lot of uncertainty and risk, and how to communicate some of those ideas back to the plant teams.

So I try to also evaluate if the skills that I currently possess are important for the industry, how much are people willing to pay for them, and also creating myself, at least in my mind, a trajectory of what I need to learn in the short and long term as to what will be the most beneficial, once again, from an ROI standpoint for myself.

The second question is extremely interesting. As many of I have taught PLC programming for many years and still mentor a lot of different engineers in the field and still do my best to stay current on a variety of different platforms. So here we have a question from the username of drygolf5000, and he's saying, "Hello.

As the title says, I have been working in a company in Europe that provides industrial automation services such as PLC programming and project planning, et cetera." So it is a consulting/systems integration company. " I'm fresh out of university where I have spent two years learning basics of schematics, PLC programming, wiring, electrical parts, et cetera, but now I am literally programming whole projects in Beckhoff, Siemens-" In function block diagrams and structured text in which I have 1% experience from school.

So my question is, how do I get better at PLC programming by myself? Yes, I did watch Jacob Segatowski playlist. It did help, but it's still need to learn all these things much deeper. Thanks all for help. So I have a couple of pieces of advice, but I will mention first and foremost that my skill set has been predominantly in North America, so a lot of what I came through was initially ladder logic.

I've only learned structured text and function block diagrams a little bit, I would say three to five years into my career, just because of the strong presence of Rockwell and the I would say choice being ladder logic as opposed to anything else in this part of the world. So I will give that as a quick disclaimer.

So he is learning projects and doing projects in Beckhoff, Siemens, in, like I said, function blocks and structured text, 1% experience from school. This is completely normal. I think that even when I, looking back, was in university, I assumed that everything we are learning is going to be highly applicable in the job market, and that is not at all the case.

If you're watching this as a student, I would encourage you to put in all the effort necessary to get the best knowledge possible from your university classes. But also to start paying attention as to what is utilized in the industry and gain at least one, preferably two skills that are highly marketable and highly, I would say, sought after in the employer's pool once you graduate,

so if you are, for example, in your second or third year and you do a little bit of research as to what the employers are looking for, and this is a perfect post for the clue, you could spend a little bit of time outside of school. I'm not saying to run two parallel programs, but spend five to ten hours maybe per week, maybe every other week to learn, let's say, PLC programming at a deeper level so that you're ready.

Now, to get back to his comment is, how do I get better at PLC programming by myself? He already watched the videos on YouTube. That is an excellent start. I would continue looking at other materials. I know that there's quite a few videos available on the Udemy side. I would also, first and foremost, again, he hasn't mentioned this, but I'm assuming that the controllers that he's using for work are usually difficult to get back into your home office to be a little bit more relaxed and less stressed on performing the actual customer requirements and spending time just playing around with maybe using different instructions you wouldn't necessarily know that well.

So I would say if you have the opportunity, either bring a controller home for learning purposes outside of work or purchase one of these controllers, invest in your work just as you, you have done with the university program. Purchase a controller, bring it back home, run the software. I believe that Beckhoff is a lot better on the simulation side, so maybe you don't even need a controller.

But I would spend the time building out some of the routines that you've talked about, but also understand what works, what doesn't work. I would also start working with external systems, so right now, he mentioned Beckhoff and Siemens, and PLC programming is not going to be just the actual control system, it's gonna be talking to HMIs, talking to a SCADA system.

Implement a small project on your own. Build something out that you can actually demonstrate but also learn from, and you will get better by putting in the work, realizing what works, realizing what doesn't work, then learning some of the mo-more advanced techniques. So we've talked about UDTs and AOIs.

Again, obviously, there are data structures and functions depending on the language of choice, but you start using them, and then you start getting better at them because you see where they work and where they don't work. You make adjustments. You also start creating and building a repository not only of knowledge, but also of assets for yourself as you navigate some of these challenges that you can then reapply at your workplace.

And once again, if you're working with a real system, you will go out in the field, you will commission the system you have built, and a lot of times you will have some leeway to realize where you need to make adjustments. And you do get better through experience, but you do need to, in my opinion, spend some time outside of the normal working hours to get the best out of the learning opportunities

hi, my name is Vladimir Romanov. I am the founder of Joltech as well as Solis PLC. With a background in electrical engineering and an MBA, and over a decade of experience leading projects in manufacturing and industrial automation, I help engineers, managers, and manufacturers make smarter technical and business decisions, modernize their operations, and build stronger careers.

If you're serious about manufacturing, automation, and staying ahead in the industry, subscribe and join the community

the next question is fairly loaded. Fair warning, this is a big career question. Again, I have mentored countless engineers and technicians through the years, so I definitely have a couple of opinions on this front. Let's take a look. So this comes from a

username GunslingerofGilead82. "Career change without an electrical engineering/electrician background. I have been looking into a career change to a skilled trade that builds on my existing strengths and experience, and have recently learned about PLC, SCADA, and industrial automation, but I need some guidance.

First off, I'm in my early-mid 40s. My experience is a bit of a mixed bag. I spent three years in the army as airborne infantry, another year and a half in reserves, then became a cop. I spent 15 years in law enforcement, with the last seven of those years in investigations. I have been working in tech IT for the past four years as a Salesforce administrator, but I have learned to code as well, and I thoroughly enjoy creating and troubleshooting automations.

I don't have a college diploma or university degree, nor do I have any trades training certifications. It's my current understanding that most people who get into this industry either have an engineering degree and background, or they were electricians by trade. Given my background, do I have a reasonable chance at getting into industrial automation, assuming I spend some time Taking some courses, learning PLC and SCADA.

Should I be trying to an apprenticeship as an industrial electrician? Are there recommended courses, program certifications that all you can recommend? I found some online courses from local college that I'm considering. They are part of a certificate program for PLCs. Finally, any recommendations on breaking into the industry and finding employment or an apprenticeship?

Any advice and direction you can give me is much appreciated. So I love these posts because we have a ton of information as to the background of this individual. We are not shooting in the dark, so to speak, with a couple of lines of description. This is a very detailed post, and I would say a great way to be asking for feedback.

So I would start almost from the end, so he's talking about pursuing a course on PLCs and a certificate in PLCs. My general perception of his background, that he is a lot more suited into a SCADA engineering, SCADA developer, SCADA integrator type of a role. And the reason why I say this is because of the most recent four years spent as a Salesforce administrator, but also programming and enjoying of creating and troubleshooting automations.

Now What you need to understand generally speaking is that you have your plant floor field devices, and those are of course your sensors, your actuators that feed into your PLCs. Then those PLCs of course are controlled by a SCADA platform and then show up some data in the MES. And of course, I'm oversimplifying.

I have multiple videos on that front. But if you have been on the Salesforce administrator side and you understand CRMs, you understand ERPs, so you're understanding business IT software, I would say that the most direct migration is into SCADA and MES because they have a very similar feel, and when you go for interviews, you can have a very, I would say, like easy way in because you're painting a picture of, " I've done a lot of IT I've done a lot of tech.

I've worked on Salesforce. SCADA and MES are a perfect fit for me as I would like to work on more automation side of things." Now, going into PLCs, this is a completely different world, and you can definitely achieve it with your background, with the background that he's describing, being in the army, working as a police officer, working in investigations.

I think it is definitely attainable. I just want to be clear on your maybe expectations and what you would like to be doing. Because the pivot from doing IT and tech as a Salesforce admin into an industrial electrician is going to be quite drastic, at least from my viewpoint. So I would spend some time understanding what it is that you are looking for.

Are you interested in being on the floor a lot more? Are you interested-- Because if you're looking to work at the plant level, this is gonna be a job where you're constantly at least in the field, you're always tinkering on hardware. It is less so programming, I would say, at least in the early years. It's gonna be troubleshooting actual equipment and electrical systems more so than programming.

Versus if you specialize in SCADA and MES, I think it's gonna be more of a, again, at the desk type of a job. You're going to be program- the systems a lot more. It's gonna be much more of a familiar environment and thus an easier transition. In terms of what to actually learn, so again, I think first and foremost, you need to decide if you really want to be an electrician and someone that is very hands-on with the equipment, or you want to have a more software-oriented role.

If you're going to pursue SCADA, I would highly recommend going through Inductive University which is completely free. You can opt out to go to their official trainings. I believe it's two to three grand for the entire week. They have restructured the trainings relatively recently, but you can learn everything you need on the SCADA side for free from the Inductive Automation website, and you can also download the software, understand SCADA at a very good, I would say, level in order to land jobs in SCADA and MES type of development.

Are you going to command a premium immediately after completing Inductive University? I would say no, but you certainly will find entry-level jobs that will accept your knowledge once you are upskilled, provided, again, that you create a good story for the fact that you have already been working in IT and tech, you've worked on CRMs, ERPs, and you have completed the certification for Ignition.

If you're really looking to become an electrician and step into that more plant-oriented type of a role, I don't think you have a choice but to learn the world of PLCs, I would say. But I almost-- I would say I would be hesitant to just learn programming. Depending on the certificate that you're considering, I would say that~ found-foundations~ again, I don't know your experience and background on that side, but foundations of electrical systems would almost be more valuable than learning how to program PLCs, because once you walk into those interviews, it will be highly unlikely that someone is looking for you to start programming PLCs immediately, especially knowing your background.

They will probably pair you up with a much more knowledgeable either engineer or technician that can show you the ropes, that can give you different work orders. You will have projects on maybe modernizing some equipment. You will always be pulled into troubleshooting activities. So I would be very cautious on what you invest your time and money in when it comes to PLCs,

and I'm not saying that you shouldn't learn how to program PLCs. I think that you should also consider and reflect on what your actual electrical background looks like. That being said, I would be very surprised that a person with your background could not find a company. A- and again, maybe you spent a little bit of time outside of work upskilling, but they would be very glad to have you because it seems that you have applied yourself, you have demonstrated a fairly good career, I would say, in other fields.

And if you're actually interested in learning control systems and PLCs and SCADA, the companies that are-- or that you should be targeting would be very happy to have you, and they will send you to trainings, and they will pair you up with the right people so that you could learn the missing pieces for your specific background.

So I honestly... My best advice is with your background, polish up your resume, make sure that everything is listed, you have a cohesive story of where you're looking to go. And I would start applying. I would start getting feedback from engineering managers, engineering directors. Ask them what it is that you are lacking.

Make sure to review the job postings. Make sure to understand what you're looking to go into. And like I said, just start having conversations with individuals hiring and filling those positions. Talk also to recruiters. There are going to be, not that many, but there are specialized recruiters in our industry that can review your profile from A to Z.

They talk to many hiring managers. They will understand what it is that you need to be marketable. And I would, again, I would be very surprised that you could not make this transition relatively easily without the need of upskilling or very extensive training before you actually are employed.

All right, the next question that we have is of a technical and philosophical nature. And so this comes from the username by Mikey382, and he's saying, "How often do you limit system functionality on purpose? Sometimes given the same budget, it is possible to expand the functionality of a system through math, clever use of sensors, signals, timing, etc.

How often do you go the extra mile? As I put in more years under my belt as a system designer, I came to the conclusion that it is just not worth it. You need to limit the scope of functionality offered to clients to only those that the budget will allow the system to perform 110% perfect. No matter how many times you make it known to the client again and again that this number is only an estimation, this position is only an approximate starting point for an operator, etc.,

etc., they will never understand even if the performance far exceeds the specification. Just my two cents, try to never offer performance to 80%, even if you can make it perform to 90%, because clients will expect performance to 100%, even if they paid for 70%." And as I've said, this is both a philosophical and to an extent a technical question.

It is something that is worth discussing because certainly in our industry If you are designing or working on highly complex systems, there is the opportunity to spend a X amount of weeks or months or years on a project that will over-deliver based on specifications. But of course, you are time-constrained, you are resource-constrained, and in most cases, there is an expectation of delivering a system that will meet all specifications within a very short amount of time,

and how do we draw that line or where do we where do we deliver that makes sense for both the customer but also for the, in this case, I'm assuming he's at a systems integrator, so both parties benefit, and so in order to understand that, we need to take a step back and understand the process,

so when you start doing project work, regardless on which side you are on, you will often see or understand that there's two ways of billing for said project. The first one being T&M or time and materials, and the sec one-- second one being on a per project basis. So what that usually entails is if you're the end user, you will reach out to integrators that are capable of meeting your scope of work or your needs when it comes to automation and other projects of course, but in this case for automation.

And you will put out a scope of work that may be three months long, maybe six months long But ultimately requires external resources that you do not currently possess internally to complete. And so the other entities like integrators, machine builders, so on and so forth, will write their proposals for a specific project that you have defined and tell you that they're comfortable with T&M or on a per project basis.

So this is where the constraints usually come in. Of course, everybody has a budget, everyone has a timeline, and you need to meet somewhere in the middle between the two parties, or sometimes in more complex projects, multiple parties, but ultimately at the high level, two parties to execute said project.

And like I said, the main criterias, at least on my side, have always been budget and time. So of course, if you're looking to get something done in three months, the integrator is going to come in and say we're going to need maybe six months worth of runway to execute this." They will then say, " We're either comfortable bidding the entire project, or you can have these resources on a usually bi-weekly basis."

So you would buy bundle of hours from them. They will show up and they will work on the project as long as they need to be there, either supporting existing operations or executing said project, and then whatever happens, so when you're talking about T&M, usually the customer assumes most of the risk.

And the reason I say that because is the engineers that are billed on an hourly basis will come in and will do their best to execute the project. But if the project goes over budget or over time, the customer has the option, they can extend those contracts, or they can simply eat the loss and whatever has been delivered in that timeframe will be what will be put on the plant floor,

so in that case, the project that is paid for to seventy percent will be delivered at seventy percent, maybe less, maybe more, depending on how efficient the engineers are If you're bidding on the full project, you now start to have a shift in balance of power, so usually the end user will put some money down towards the integrator to start on the project.

But as the project reaches the total amount of hours it has been billed for, you will start having pressure on both sides. The end user, of course, is looking for one hundred percent completion. The integrator is trying to articulate that they need more capital if they were not able to meet some of those standards.

And of course, the final result of that project will probably fall somewhere in between. Sometimes negotiations will extend the project, if we're overrun by ten percent, usually it's fine. If we're under run by ten percent, usually it's fine. But if the project requires double the capital than originally anticipated, that could be a massive problem on both sides, to be honest.

Either the SI eats the cost or the manufacturer eats the cost. So now the question becomes, as the integrator as the system designer, where do you draw the line? Generally speaking, if you are assigned a project by upper management that has negotiated the sale, you will generally have a timeline.

And depending on how big or how small the systems integrator is, you will have somebody breaking down the different milestones as well as the key deliverables and different features for that specific project. And those features could be as small as we need to finalize this specific SCADA screen, we need to make sure the panel design is built, we need to make sure that we release the CAD drawings.

So it could be a waterfall chart or a Gantt chart, if you're familiar with that terminology, on the different milestones. Now, the question here is: how often do you go the extra mile? For me, the extra mile, to be justified, needs to be understood by the customer, and when I say understood is number one, we have the budget to go the extra mile, meaning that we have some perhaps leftover hours or capability or something else that is allowing us to go the extra mile in terms of time and billable materials and resources.

But like I said, the customer understands why we are delivering the extra mile, and there is a tangible benefit for us as the SI to delivering that extra mile, and it... Of course, it depends what the extra mile is. It could be something as I've seen very, for example, rough HMI screens recently, and I thought to myself, did they not have the extra one or two hours to just make nicer UI?

Maybe they didn't. But ultimately, is that extra mile worth it for the customer for an HMI that's going to sit up a ladder in an environment where it rains, where it snows? It doesn't necessarily matter for them, so they will not appreciate or even care about that extra mile.

However, if you have features, for example, again, referring to some of my recent experience where you have a SCADA system, maybe the extra mile, which was not part of the scope, is adding some of the trends and data storage capabilities that allow the cu-customer to track the batch and thus realize further savings through the system.

If you can explain to them and clearly articulate why this has been done and that it has been done outside of the scope, just out of pure, I don't want to say generosity but at least to some extent, reciprocal maybe behavior in the future, I believe there is a tangible benefit. Of course, you also need to evaluate what-- how much time does it take for you to implement this feature, and should you be charging for it?

And again, there's a lot of debates and gray areas in the space, but I think that almost always this conversation and this question should begin with discussion with the customer. The customer needs to understand what is within scope and what is out of scope. Because I have been part of these conversations as well, where you over-deliver and the component that you have over-delivered on ultimately suffers, ?

So again, you have not spent all of your time polishing and making sure that specific sustem- subsystem works. It does work, but it maybe doesn't work to 100%. And now, once it is part of the actual project and system, the expectation is st- is that every single component delivers tremendous value. And sometimes you get complaints on aspects and on features that have not even been part of the scope.

So make sure that the customer is very clear. Make sure that they understand what is within scope, what is outside of scope, and what they have or are able to make complaints on once the system hits the plant floor

We've got a last question that is of a technical nature, but is very topical when it comes to OT networks. As many of I have a lot of different content and experience when it comes to OT networks, so always open to have a conversation and helping out on that front. That being said, this question comes from the user named by KingDentist2139.

IP address conflict. Came into work today with an IP conflict. Over half of our facility is behind NAT, network address translation modules, but today our network had a conflict and determined that two Rockwell devices were addressed the same. DHCP showed both were Rockwell devices. There's nothing I can fathom that would or could change the IPs.

I checked every device on our network, but couldn't find a duplicate. No power outage or device change, but it happened. We gambled and thought DHCP changed the other MAC address to an unused IP. Any ideas? Because it caused many hours of downtime. So my in- initial comment is there's very little information to go on.

We don't have the IP network map. There's a lot of questions on my side as to why half of the facility's behind NAT modules. I have a question as to why DHCP is being used in a OT or production environment. This is not a usual setup, at least from what I have seen in the field over many years.

And so I have more questions than I have answers for this specific use case. We also don't know which devices have the same IP addresses, so we have two Rockwell devices were addressed the same. So my original thought is, of course, addressed the same, I would assume that if he's using a lot of NAT modules, that many of the Rockwell hardware in different cells and different lines will have the same IP address, and the reason why you even need NAT in the first place is to translate those field IP addresses into what the plant understands.

The first suspect for me, it would be the NAT modules, ? So depending on how it is done, you can use a fully managed switch. So for example, here I have a Stratix 5700 series. I don't believe that this is the fully featured version, but you can find a version that will do network address translation.

The problem you can run into, of course, is you have all the same physical ports, but the way you address the ports for NAT is going to be different. So if you move any of the cables, it is possible that the devices that were behind the NAT have now been put on the plant floor side, depending on how the switch has been configured.

So I would start tracing back to whatever NAT modules they are using. Of course, there's also different flavors of NAT modules. I have seen ones that only have two ports, in which case this scenario could not be possible. The second question is, as I've said, what kind of Rockwell devices are you using? In almost all cases, I would say on newer hardware, you should not run into a problem where the IP address is reassigned.

However On older devices especially the ones that are usually supported by a battery that needs to be replaced on a regular basis, what you will find is that the IP address may reset with a power outage, with any, I would say, like reset of power of the program. So perhaps the IP addresses had defaulted to Boot P DHCP, in which case, because you have a DHCP server, it had assigned them to the exact same IP addresses.

And once again, this comment goes back to the switches as well, because those switches will, in certain cases, pass through your trunking port DHCP type of assignments and allow those devices to be accessed or assessed via other locations. So those would be my initial thoughts. Of course, to further diagnose, I would need to see the entire plant network diagram.

So he says that he checked every device on the network but couldn't find a duplicate. Once again, has this problem been resolved? Is this still a problem? This is a very strange, I would say, troubleshooting log as well, because from what he's saying that nothing happened and the addresses just changed.

He's-- he mentions that no power outage or device changed, but it happened. So again, what I'm being told this by usually different teams in different facilities, I am extremely skeptical, so sometimes you will have a blimp in the system that no one will necessarily notice or log, but it is there.

So sometimes the operator on the night shift wanted to replace, let's say something on the machine. They move the photo eye, they inadvertently shut off the panel, which they shouldn't have. They power on the panel, and that's how we've lost... because no one has replaced the battery, we have lost the IP address, and now we're in this situation.

So there's, of course, a lot of ideas. I think that it's very difficult in these scenarios if we haven't found a duplicate to trace back exactly what had happened. That being said, this is also very suspicious because we say that the network had a conflict and determined two Rockwell devices were addressed the same, yet we could not find a duplicate,

so again, this is extremely strange. How did you find the two devices? Usually, it should tell you what the IP addresses were. It should tell you at least or give you an indication of the location. So further troubleshooting is definitely required. I can think of a couple of scenarios. It is very hard to say based on the information provided.

In any case, that is the last question that we had for the week. Thank you so much for tuning in. As always, leave us some questions, leave us some comments. Appreciate all the feedback from the community.

ShareLinkedInXEmail

Keep listening