This system is pretty large, so I know this initial writeup won't cover everything. Please feel free to ask questions about anything on the system though. Besides the Phidgets drivers, I used two jquery libraries - I developed the rest of the system on my own in .Net
I intend to continue answering questions after the contest closes as well. If the thread or board get closed as the contest ends, feel free to send me private messages.
PACA - Production Application for Control Anywhere (I name my software after animals, I have a zoo).
The intent is to provide a mechanism to perform production data gathering and accurately track flow, specifically the speeds at which items move through the floor.
As similar systems are likely to be added to my project pipeline, it was a personal goal to make the project as flexible as possible; reducing future development costs.
PACA consists of a few servers working together, and collections of Phidget items on the production floor.
Typically a small collection of Phidgets will be linked with a physical work station and process. Based on floor plan, groups of these stations are plugged in to Phidget SBCs, which are wired in to the network. An always-on application server handles the "plugins" for each station, starting and stopping them as required, and managing all physical interaction with the station. These plugins also log data to a database, such as time per part. An ASP.net site serves as the primary point of contact for management and administration, displaying aggregate data from the stations, and providing screens to control and modify individual stations.
Phidgets were chosen due to the cost, flexibility, and incredible ease of development. Other systems exist which were on-par in one or two of the areas, but Phidgets stood out as having all three.
A typical station consists of:
- An LCD panel with IFKit (1203_2) to output messages to operators and handle all sensor and LED signals.
- One or two RFID readers (1024_0 or 1023_1 on older stations), used to track operator presence to improve accuracy.
- One analog sensor, used either as a manual trigger - such as a touch sensor in a 3d printed enclosure - or hands-off by monitoring a piece of equipment - such as a photo eye watching a linear slide.
- A few LEDs, used to output various information based on color and blink patterns. For example, a blinking red light in the sensor enclosure means that the operator forgot to place their RFID card on the station.
- Miscellaneous USB devices - hub, cables, etc. These are dominantly used so a station can be set up by plugging one cable in, instead of two or three.
The application server is running a Windows service. This service checks a database to determine which stations should be on or off, as well as monitoring the overall health of the stations. When a station needs to turn on, the service launches a separate process which loads a "plugin" containing code specific to the type of station it is controlling. Launching separate processes was chosen as, on occasion, system stability issues would cause a station to go down; when they are under a single process, this could cause one trouble station to take down the whole system unexpectedly.
By using the plugin system, it allows two executables - a service and worker - to handle any conceivable station configuration without being changed - as new configurations would just need to make use of the plugin interface, and are otherwise autonomous.
Within each plugin is the code that both runs the station - handling sensor and RFID events, and outputting messages via the LCD panel and LEDs - as well as logging events we want to track to a database for further analysis.
The data is stored in a relatively raw format, consisting of timestamps and event properties such as quantity and operator name. From there, a calculator object runs the analysis on requested data ranges to output things like the average time per part, or total number of hours the station was active. By using this method, instead of storing calculated values, all changes to calculation methods are automatically retroactive - keeping data points for any given range accurate when compared to current values, and flexible to the business needs.
By deploying to a single app server instead of individual SBCs, it also made for very fast, easy, and consistent deployment and development; without sacrificing any flexibility in station configuration.
General notes and conclusions:
While the system design and setup has been time consuming, it is already returning useful data. It also allows for potentially more accurate time studies, as workers move at a more natural pace instead of getting distracted by the "figure with a clipboard watching over their shoulder".
Overall the workers have been supportive of it as well. Even though it is "always watching", it has allowed them to immediately see when they're getting better; in some cases leading to an urge to race their own times.
For general numbers:
- PACA currently consists of 4 stations on the production floor, 3 of which have sensors embedded in equipment (see photo eye in video). An additional 35 or so stations are in process for other areas on the floor.
- In practice, the SBCs have been stable with up to 4 stations with 1 RFID reader each; or two stations with 2 RFID readers.
- The application for an individual station runs around 12Mb of RAM, with the monitoring service at about 10Mb of RAM. Depending on when I'm watching, total usage runs between 30-60Mb (with 0-2% CPU usage).
- To date there have been 22,777 log entries made, with an average of hundreds per day.
- 18 objects spread between 6 executables and dlls.
- The server applications consist of about 3/4 the total code, with the website filling out the rest.
This is a small snippet of a data download from PACA. The first "taps" for a 0 quantity are when the operator came to the station - spoofing a tap record makes the day-long calculations a little easier. The Downtime record is when the operator came back to the station, with the "detail" being the number of milliseconds they were gone - looking at the previous record for station 10 tells us that the operator put their card down, then immediately stepped away for about 12 minutes before coming back (note that the CSV timestamps only have minute resolution - actual data is stored down to the second).
The result of this is that PACA will subtract 12 minutes from the total time when it calculates the time since the last tap.
As station 7 doesn't have any downtime, it would show a total of 4 units in about 6 minutes, with PACA outputting about 1.5 minutes per unit.
The total time taken is automatically adjusted if a second operator is present - denoted by an entry under the badgePresent2 column for the record.