pctechguide.com

  • Home
  • Guides
  • Tutorials
  • Articles
  • Reviews
  • Glossary
  • Contact

Monitoring in Machine Learning Part 2: Monitoring Techniques

We talked about the reasons that you need to monitor in machine learning in our last post. We are now clear about the main factors that can degrade the performance of a model.

So we can define monitoring as the phase of Machine Learning Operations in which we measure different performance variables of the model and compare them with reference values to determine if it continues to generate adequate predictions or if it is necessary to take actions to improve performance.

And there are several ways to perform this monitoring, some quite simple and others more sophisticated.

Monitoring through global metrics
The simplest of all is to continuously record a global metric of the model’s performance and compare it to a reference level.

For example, if we have a face detection system that at the development stage had an accuracy of 97% then we can periodically (e.g. daily) record this performance on the deployed model and if it is observed to fall below this reference level an alert could be generated indicating that we should take some action before things continue to get worse.

The drawback of monitoring using a global performance metric is that we cannot determine the reasons behind the degradation, i.e. whether the underlying problem is “data drift” or “concept drift”.

Monitoring through statistical methods
A more sophisticated way to perform monitoring is for example to obtain the statistical distribution of the input data before deployment and periodically calculate this distribution but for the data used by the deployed model, and then apply a statistical test to determine if there are significant differences between one and the other. In the case of finding differences we could conclude that the origin of the degradation is in “data drift”.

Monitoring in detail
Perfect, we already have clear the main factors that can degrade the performance of a model.

So we can define monitoring as the phase of Machine Learning Operations in which we measure different performance variables of the model and compare them with reference values to determine if it continues to generate adequate predictions or if it is necessary to take actions to improve performance.

And there are several ways to perform this monitoring, some quite simple and others more sophisticated.

Monitoring through global metrics
The simplest of all is to continuously record a global metric of the model’s performance and compare it to a reference level.

For example, if we have a face detection system that at the development stage had an accuracy of 97% then we can periodically (e.g. daily) record this performance on the deployed model and if it is observed to fall below this reference level an alert could be generated indicating that we should take some action before things continue to get worse.

The drawback of monitoring using a global performance metric is that we cannot determine the reasons behind the degradation, i.e. whether the underlying problem is “data drift” or “concept drift”.

Monitoring through statistical methods
A more sophisticated way to perform monitoring is for example to obtain the statistical distribution of the input data before deployment and periodically calculate this distribution but for the data used by the deployed model, and then apply a statistical test to determine if there are significant differences between one and the other. In the case of finding differences we could conclude that the origin of the degradation is in “data drift”.

Or we can do something similar but for the data distributions at the model output before and after deployment, so that if we find statistically significant differences we can conclude that the performance degradation is in this case due to “concept drift”.

Conclusion
Very well, in this article we have seen that after deployment it is very likely that the performance of the model begins to decline and this is precisely because both the data and the environment in which the model is in are dynamic and can continuously present variations.

So monitoring allows detecting this performance degradation, either by analyzing global metrics or by using more advanced techniques such as the use of statistical tests applied to the model’s input or output data.

But this process does not end with monitoring, because if performance degradation is confirmed, corrective actions must be taken to keep the model in production. This phase is known as model maintenance and will be discussed in a future article.

Filed Under: Articles

Latest Articles

Windows Antivirus Adviser

Windows Antivirus Adviser is a fake antivirus program.  This kind of virus is very common on the internet and they have been around for years under many different forms. People normally get infected with these fake antivirus programs after downloading free programs that have this bundled with the … [Read More...]

dymo-250-turbo-label

DYMO Labelwriter 450 Turbo Label Printer

The DYMO Labelwriter 450 Turbo Label Printer can print speeds up to 71 labels per minute. What I really like about this thermal printer is you can use the included DYMO Stamps software and print accurate USPS approved postage. There is no membership fee to use the program which makes it a no brainer … [Read More...]

AMD Thoroughbred

In the summer of 2002 AMD began shipping its first desktop processor built using a 0.13-micron fabrication process. AMD expects the transition to the finer process technology - the Thoroughbred core is a minuscule 80mm2 compared to its predecessor's 128mm2 - to deliver improved performance, lower … [Read More...]

Why Internet Bundles Are the Next Battleground for Gamers

A weak internet connection can turn seamless gaming sessions into frustrating experiences filled with lag, buffering or sudden disconnects. For … [Read More...]

Importance of Inbound Marketing in the Digital Age

A couple of months ago, Zacks reported that Hubspot was starting to make some major changes to its inbound marketing strategy. They talk a lot about … [Read More...]

Damage Control Strategies for Resolving Online PR Crises

Last July, Astrologer faced a major crisis after its CEO went viral at a ColdPlay concert when having an affair. This was just one of the many times a … [Read More...]

AI is Not Killing Computer Jobs Like Doomers Projected

There is no denying the reality that AI technology has played a massive role in disrupting our lives. A growing number of people claim that AI … [Read More...]

Everything You Need to Know About Sourcing Circuit Boards From U.S. Suppliers

In This Article This article includes: Why Source PCBs From the United States?How to Get a Quote From a U.S.-Based PCB ManufacturerThe Top U.S. … [Read More...]

Top Taplio Alternatives in 2025 : Why MagicPost Leads for LinkedIn Posting ?

LinkedIn has become a strong platform for professionals, creators, and businesses to establish authority, grow networks, and elicit engagement. Simple … [Read More...]

Guides

  • Computer Communications
  • Mobile Computing
  • PC Components
  • PC Data Storage
  • PC Input-Output
  • PC Multimedia
  • Processors (CPUs)

Recent Posts

Glite

During 1997 a number of companies were independently working on versions of splitterless DSL. The theory behind … [Read More...]

Don’t Jailbreak Your iPad Until You Read This

People have been jailbreaking iOS devices for quite some time in an attempt to make them more useful for their specific needs. Sometimes, they find … [Read More...]

BitDefender Antivirus Pro 2015 Review

Pros : Has a ton of new useful features over previous versions. One of the best antivirus protection systems around. Offers a secure browsing to … [Read More...]

[footer_backtotop]

Copyright © 2026 About | Privacy | Contact Information | Wrtie For Us | Disclaimer | Copyright License | Authors