Update README.md

README.md

@@ -15,22 +15,32 @@
 </div>
 
 # Ylem
-The open-source data streaming platform is built on top of Apache Kafka.
+The open-source data streaming platform is a one-stop-shop solution for orchestrating data streams on top of Apache Kafka, Amazon SQS, Google Pub/Sub, RabbitMQ, various APIs, and data storages.
 
 # Dependencies
 
-1. Install Docker 4
+To be able to run Ylem locally, the following dependencies need to be installed and configured.
 
-2. Install Apache Kafka
+## 1. Docker 4
+
+If you don't yet have Docker 4 installed, [install](https://www.docker.com/products/docker-desktop/) it from their official website for your OS.
+
+## 2. Install Apache Kafka
+
+Ylem uses Apache Kafka to process pipelines and tasks. [Install](https://kafka.apache.org/) it from their official website or skip this step if you already have it installed.
+
+We also recommend installing the Apache Kafka GUI software of your choice to manage its topics and subscriptions.
+
+## 3. Create Kafka topics
+
+To be able to work correctly, Ylem requires the following Apache Kafka topics to be created:
 
-3. Create topics manually
 * task_runs
 * task_runs_load_balanced
 * task_run_results
 * query_task_run_results
 * notification_task_run_results
 
-
 # How to start with Docker
 
 ## Start all containers
@@ -56,6 +66,34 @@ E.g.
 ``` bash
 docker compose build --no-cache ylem_users
 ```
+
+# Folder structure in this repository
+
+Ylem is a set of microservices. Each microservice is represented by one or more containers in the same network and communicates with each other via the API.
+
+``` bash
+|-- api                  # api microservice
+|-- backend
+|--|-- integrations      # integrations with external APIs, databases and other software
+|--|-- pipelines         # pipelines, tasks, connectors
+|--|-- statistics        # statistics of pipeline and task runs
+|--|-- users             # users and organizations
+|-- database             # a container for storing databases for all the microservices
+|-- processor
+|--|-- python_processor  # processor of the Python code written in pipelines 
+|--|-- taskrunner        # task runner and load balancer
+|-- server               # Nginx container in front of all the microservice APIs allowing to avoid CORS issues on the UI side
+|-- ui                   # user interface
+```
+
+Each microservice has its own README file containing more information about its usage and functionality.
+
+# Documentation
+
+The user and developer documentation of Ylem is available at https://docs.ylem.co/.
+
+The [open-source section](https://docs.ylem.co/open-source-edition) contains information about the [task-processing architecture](https://docs.ylem.co/open-source-edition/task-processing-architecture) and [configuration of integrations](https://docs.ylem.co/open-source-edition/configuring-integrations-with-.env-variables) using .env files and parameters.
+
 # Key contributors
 
 * [olschaefer](https://github.com/olschaefer)