Visor: Real-time Log Monitor Documentation

Introduction

Visor is a Real-time log monitor. It is built on Apache Spark, utilizes Spark Streaming to stream the log files, and uses Spark ML for Machine Learning. It supports monitoring multiple log files at the same time, with the help of Apache Kafka.

About This Book

This book is a document for project Visor, and it's also an example of how to build a scalable real-time log monitor, it shows most of the details of the applications' structure and codes.

About The Author

Written by Wenyi Xu.

• My Github
• My LinkedIn
• My Email: [email protected]

Directory Structure

config/
script/
src/    
    fake_log_gen/
    model/
    mini_monitor/
    socket/
    kafka/
test/
README.md
requirements.txt

Source Code

Source code can be found in https://github.com/xuwenyihust/Visor.