-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
0b459fc
commit 4ae2921
Showing
1 changed file
with
9 additions
and
57 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,73 +1,25 @@ | ||
| --- | ||
| title: "Database support" | ||
| title: "Databases" | ||
| format: html | ||
| editor: visual | ||
| --- | ||
|
|
||
| One of the main tasks of Hal9 is to enable users to quickly and easily create applications and conduct analysis on databases. When we have the data stored in csv files we can follow a similar approach to what was covered in the preceding tutorial to analyze it, but what happens if we have combined databases hosted in a cloud environment? or if we want to extract data from an application such as Pilot or Notion, create combined databases and analyze them in Hal9? | ||
|
|
||
| Hal9 currently does not provide direct support for fetching data from external applications; however, it does offer a direct method of parsing this type of data using Postgres, which is what we will be covering in this tutorial. | ||
|
|
||
| ### Connecting to a database warehouse using Hal9 | ||
|
|
||
| Hal9 allows the users to connect and query a database in Postgres. Not only this, but it also gives you the necessary credentials to create your database in this data warehouse. | ||
|
|
||
| The steps to do it are: | ||
|
|
||
| - Once you are logged in, you must go to settings and you will notice that the *Enable database support* option is not selected. | ||
|
|
||
| <center><img src="tutorial_02/120.png" style="max-width: 70%"/></center> | ||
|
|
||
| - When you select it, a *Create* button will appear, which when pressed will give you your Postgres credentials, as it is shown below. | ||
|
|
||
| <center><img src="tutorial_02/121.png" style="max-width: 70%"/></center> | ||
|
|
||
| These credentials give you access to a database in Postgres, which you build by connecting it to other applications where you will extract data from using differents integration frameworks. In case you do not have a technical background, we recommend using [Bracket](https://www.usebracket.com/) for this step. | ||
|
|
||
| <center><img src="tutorial_02/1.png" style="max-width: 70%"/></center> | ||
|
|
||
| [Bracket](https://docs.usebracket.com/introduction) is a fast, code-optional way to sync data between databases, data warehouses, and SaaS tools and enables you to set up limitlessly flexible one-way or two-way syncs between any pairing of database or SaaS tool. It support differents integrations, for example Google Sheets, Notion, Snowflake, and Postgres. | ||
|
|
||
| Once you are connected to your database, you can start to chat with your data! | ||
|
|
||
| #### Example: Notion \< Bracket \> Postgres \<\> Hal9 | ||
|
|
||
| Once we have the Postgres credentials that Hal9 gives us and a database on Notion, it is very easy to connect them both using Bracket as an intermediary. To do this we will use the [Bracket legacy app](https://legacy.app.usebracket.com/). | ||
|
|
||
| In this app, we select Notion as the main data source and then select the database we want to use. In case you do not have any available database, we must select Link new bases and give permission to Bracket to access them. | ||
|
|
||
| <center><img src="tutorial_02/122.png" style="max-width: 70%"/></center> | ||
|
|
||
| <br> | ||
|
|
||
| Then we select Postgres as the secondary data source and introduce the credentials we obtained from Hal9. | ||
|
|
||
| <center><img src="tutorial_02/123.png" style="max-width: 70%"/></center> | ||
| Hal9 currently does not provide direct support for fetching data from external applications; however, it does offer a direct method of parsing this type of data using PostgreSQL and Snowflake but we are currently adding a lot of different database connectivity solutions. | ||
|
|
||
| <center><img src="Images/data.jpeg" style="max-width: 70%"/></center> | ||
| <br> | ||
|
|
||
| Once this information has been introduced, we can check the connection and if it is successful, we can proceed to the next step, in which we will select the synchronization options. | ||
|
|
||
| <center><img src="tutorial_02/124.png" style="max-width: 70%"/></center> | ||
|
|
||
| <br> | ||
| If you prefer a video explanation, we recorded [this video](https://www.youtube.com/watch?v=DRitv3r-I-s) for you. Otherwise, keep reading along. | ||
|
|
||
| In the next step, we will review our data, the connection and the mapping established between the two databases. We will also have the option of running a test run where we will not synchronize any data, run a single run where we will only synchronize our data once, or set up the connection and run the synchronization every certain time. With the completion of this step, we will have our data ready in Postgres and these will be accessible from Hal9. | ||
|
|
||
| As an example dataset, we use the *startups-hiring-2023* Kaggle dataset which contains information about startups and the jobs they are offering. If we enter the prompt *Which startups are hiring engineer?* in Hal9, we will get an app that shows us a table with the required information. | ||
| ### Connecting to a database warehouse using Hal9 | ||
|
|
||
| <center><img src="tutorial_02/125.png" style="max-width: 70%"/></center> | ||
| You can enable database connectivity in Hal9 from the Settings menu, notice this functionality requires a [Pro Plan](https://hal9.com/plans). You can then choose your databases and fill your database connection requirements. | ||
|
|
||
| <center><img src="Images/database.jpeg" style="max-width: 70%"/></center> | ||
| <br> | ||
|
|
||
| You can check this example app [here](https://hal9.com/apps/startups-hiring-2023-example). | ||
|
|
||
| ### Important concepts | ||
|
|
||
| - **Databases** are the systems used to store and manage digital information. These systems store data in an organized way and let users query it in order to extract important and relevant information. They are designed not only to store and manage data, but also to enable access and sharing of that information. They are used to store data for a variety of purposes, including recording transactions, managing inventory, supporting customer relationships, and making data-driven decisions. | ||
|
|
||
| - A **data warehouse** is a centralized repository of organizational data, used to combine databases from multiple sources for reporting and analysis. Data warehouses can be used to store current, historical, and long-term data. They can be built using physical hardware or hosted in a cloud environment. Examples of data warehouses include Amazon Redshift, Google BigQuery, Microsoft Azure SQL Data Warehouse and PostgreSQL. Data warehouses are used to gain insights for improved decision making. Common use cases of data warehouses include sales analysis, marketing analysis, customer feedback analysis, product recommendation engines, fraud detection, resource planning, and forecasting. | ||
|
|
||
| The purpose of a database is to collect real-time information for immediate use and for an application's records. On the other hand, the data warehouse collects, stores, and analyzes large volumes of historical data, for the medium- and long-term. The warehouse is the database of databases, integrating different sources of information and creating a super business intelligence system in the process. This enables much more reliable long-term decision making by having access to more data. | ||
| Once you are connected to your database, you can start to chat with your data! | ||
|
|
||
| - An **integration framework** is a structure designed to make it easier to link up different applications, data sources, APIs, and services within an organization. It simplifies the exchange of data and components between different systems, databases, applications, and services, and allows them to interact with each other automatically. This reduces complexity and costs associated with integration across different systems. Integration frameworks also provide security and automation for the regular exchange of data, and help organizations ensure the accuracy and consistency of data across applications. Furthermore, they give organizations a unified view of their distributed systems, and the ability to quickly build and deploy new applications and services. | ||
| By using databases instead of CSV you can expect better support. Their data types, table relations and processing power allows Hal9 to build data analysis faster and better. |