# Introduction to Tomat Open, analyze and automate Excel and CSV without code. ## Getting Started
Product Overview/pages/eGHfyIhbAmxqVkhnkpBg
Product Updates/pages/lLogu8LUER3vBeHlLVwM
FAQ/pages/aKyfgJv7kqym7vT7acD8
## Basics We've put together some helpful guides to get set up with our product quickly and easily. {% content-ref url="/pages/0cU5od3vh8ngrMltWpNG" %} [Exploring Data](/product-overview/exploring-data) {% endcontent-ref %} {% content-ref url="/pages/okvcei2829F1fzLqTEMh" %} [Designing Flows](/product-overview/designing-flows) {% endcontent-ref %} {% content-ref url="/pages/GA4CMZq69rolBipMpeCi" %} [Building Reports](/product-overview/building-reports) {% endcontent-ref %} # Product Updates Welcome! You can find the latest product updates listed down below by their release dates. ### **June 11, 2025** **AI Columns Node (Use AI)** * Added support for multiple output columns. * Output columns can now be generated directly from the prompt. * Added web search option for AI enrichment. * Added model selection for AI tasks. **Table View** * Added the ability to insert a new column with popular actions directly from the table view. ### **May 8, 2025** **Smart Actions** * Added smart suggestions for different column types, especially to improve working with Arrays and Objects. **Bugs** * Some major bugs in the Team version were fixed ### **March 15, 2025** **Data Catalog** * Create and switch between multiple **views** of the same dataset * Each view supports its own filters and settings * Filters now apply to the **entire dataset**, not just the sample **Other Updates** * Improved UX across the **Catalog** and **Connectors** pages * Better navigation in the **toolbar** * Updated **home screen**, toolbars, and left-side navigation * "Convert to Column" added to the **JSON viewer** * Support for "Remove All Columns" option in the **Columns node** ### **November 5, 2024** #### More updates * Improved application performance * Accelerated data flow creation and editing * Updated interface and optimized page load times, including catalogs and connectors * Improved stability of Postgres database connections during extended use * Fixed file upload and preview errors, ensuring smoother operations * Fixed issues with exporting charts to PNG/SVG * Addressed bugs related to row count discrepancies in unions and property grids ### **August 1, 2024** * Added the ability to include tables and charts from the repository, improving the visual representation of data in reports * Enabled sharing of connectors and files from the catalog, facilitating better collaboration and resource sharing among users. * Various bug fixing and improvement for Teamwork ### **April 10, 2024** * Multiple improvements in the Reports designer * The new feature lets users select source nodes from the Catalog easily * Introduced auto-formatting in API node for better usability, readability * Minor UI adjustments with updates to colors and user experience enhancements * Enhanced app performance with fixes for key bugs and issues for smoother operation * Enhanced Windows performance with C++ library in installer and streamlined access to app version info in the Help menu **Known issues** * If a local file is used as the Source node, the results of the Chart nodes will not be available on the Job page. ### **December 7, 2023** #### **Dashboards** Build visually compelling dashboards to track project KPIs and metrics. View reports as workflows in the same interface, detailing raw sources and preparation steps. #### **Flow history version** Easily navigate through historical versions of your workflows to understand their progression without losing important past changes. #### **Expression editor popup** If you input numerous expressions, expand the editor to open a popup for a clearer view of all the applied formulas. #### More updates **Group filters** You can group filters, put them in bunches, and link these bunches. Doing this helps organize data in more detailed ways, making it easier to analyze and work with complex sets of information. **Source files are now added to the catalog** It means you can keep all the files that were used in the workflow as sources. So you can check the date when these files were added to get rid of mess, and efficiently organize your work process. **Added new function for dateTime conversion** Tomat enables the transformation of UNIX timestamps into easily readable date and time formats. ### **October 27, 2023** #### **UI Updates** New toolbar design: tools are now grouped by meaning. Now, if you drag-n-drop columns in the table, rename or delete columns, only one instance of the Columns node is created. #### New table header. * Open the table on full screen with one click * Table filters are easier to find and apply ![](/files/NR9WSM3OlEjVvwaXXx9G) #### Enhanced JSON viewer Open Objects and Arrays in the structured view. Copy paths to objects and values with one click. ![](/files/R7FxotgoK0zMc4z32KtP) #### **Reworked Jobs page** For each flow, only one Job is created. Under the Job, you will find all run instances "Runs" with their logs and output datasets. Outputs from the last run are shown on the top of the Job pages. #### **AI Table node - beta** A new node where you can ask Tomat to join tables, filter, sort, or calculate fields with natural language! You can work with one or multiple tables as sources.

AI Table Preview

### **September 6, 2023** **Major UI Updates - New Look and Feel** We've given Tomat a fresh new look and feel! Our updated user interface is designed to enhance your experience and make navigation smoother than ever before. **Generate datasets with AI** Say goodbye to manual data entry! Tomat now allows you to create an "Empty table" with the help of AI. This feature streamlines data organization and reduces manual effort. **"API Call" Node** With the all-new "API column" node, you now have the power to seamlessly integrate external APIs into your workflows. This exciting addition opens up a world of possibilities for automating tasks and fetching real-time data. ### **August 2, 2023** **Features** * A new linear chat was added. * Now you can download your chat in PNG or SVG format * We also added tutorials for smooth onboarding Some major bugs were fixed. ### **July 13, 2023** **Meet our new updated design!** Even more intuitive and stunning. Wait for the next updates soon!

New Home Page

#### Small changes and issues * Filters are now case-independent * You can close the right panel on Explorer and Design Flow pages * Bar charts now have gridlines * Fixed major bugs with filters on the Explorer page after applying Group By command ### **June 28, 2023** #### **Home page** The new home page features quick links to your most recent datasets and flows for convenient navigation. #### **Magic column node (GPT)** Introducing a new node that allows you to generate a column based on text prompts. Cleanup, restructure, or enrich your datasets at a speed 5x faster with our AI's assistance. We're working to incorporate more AI support, stay tuned… ### **June 9, 2023** #### **Catalog and Explorer pages** * You can now open and analyze your CSV or Excel files without the need to create a new flow. Open them directly from the Catalog page and explore them with custom filters * Features include: filtering, sorting, grouping by, columns hiding, and data bars * You also can change the size and type of the sample data for large datasets * You have the option to save your filtered datasets or transform them into data flows for advanced operations. ### **May 24, 2023** #### **Empty table node** The empty table can be used for creating quick samples or dictionaries. Seamlessly copy and paste data from different sources like Excel or other spreadsheets using Cmd-V \ Ctrl-V commands. #### **Chat node** Now working in Runtime over the whole datasets ### **March 31, 2023** **Enhanced CSV File Support** * Work with CSV files of any size (flows continue to work with samples during design time) * Download sample data as CSV from any node during design time * Improved CSV delimiter autodetection **Refined Autocomplete in Expression Editor** * Added descriptions for function parameters * Included examples for each function * Streamlined user experience **Chart Node (Beta)** * New chart node supporting bar and column charts * Currently functional only during design time **New Materializations Types for Database Tables** * Update - add or update all new values in the existing table * Incremental updates - add or update filtered values in the existing table **Improved Source Logic** * Filter rows and select columns directly in the Source node for database tables **Node Groups** * Duplicate groups via the context menu * Enhanced canvas layout logic for expanded groups **Additional UX Enhancements** * Updated Join and Output nodes user interface * Displayed a thousand separators in the table for numerical values ### **January 5, 2023** **Canvas** * **Updated logic of canvas** * Drag-n-drop nodes without their selection * Zoom in and out * Mouse: Hold down ⌘ Command (Mac) or Ctrl (Windows) and scroll the mouse wheel up to zoom in or down to zoom out. * Trackpad: Pinch two fingers together to zoom out or stretch two fingers apart to zoom in * Hotkeys: Zoom in: Shift +, Zoom out: Shift -, Zoom to fit: Shift 1 * **New controls panel on canvas** * In the bottom right corner of the Canvas, hover over the “centralized” icon * A menu will open with buttons * to centralize the canvas * to show Minimap * to create “Text” (Memo) * **Groups of nodes** * Select multiple nodes with “Shift” + Mouse Drag * In the Action panel: select “Group nodes” (or press “G” button) * Use a group’s context menu to ungroup * Change group color from a palette on the group’s upper right corner * **Editing node names on Canvas** * Use the context menu on nodes to edit the node’s name right on the canvas * As an alternative, you also can edit node’s names from the property grid * **Copy nodes from the context menu** **Discovery & Analyze of Data** * **Statistics in columns headers** * Ranges and valid-missing percentage * for valid\missing values statistics to see → hover on a cell with ranges or unique values under column names * Histograms and the most popular values * hover on any cell right under the column’s name and click it to expand the statistics * **Statistics in preview mode in a property grid** * Filter, Duplicate, and Change type nodes show statistics like as many rows were filtered * **Change type node →** we now highlight cells that were not converted * **Show special symbols in the table** * only by hotkey → Ctrl + 1 **Misc** * **Improved UI for expression editor** * show functions’ parameters description * **User-defined scalar functions** * now you can define custom functions and reuse them in expressions #### **Known Issues** * XSLX files are not working in the flows where there are DB tables or views (neither as Sources nor as Outputs) * Canvas default cell sizes are changed * Statistics are not always loading in column headers * Groups cannot be nested # Beginner's Guide A Beginner's Guide to Data Analysis in Tomat: Connecting Data and Exploring Insights When it comes to data integration and analysis, Tomat stands out as a user-friendly platform that simplifies the process of data analysis and exploration. In this blog article, we will walk you through the initial steps of connecting your data and exploring valuable insights using Tomat. Let's get started! ### Tomat Interface In our **Catalog**, you can store files that you work with. Remember that Tomat is a desktop app, which means no data will leave your computer and appear somewhere in the cloud. We prioritize security the most. If you do not have a data warehouse, our Catalog will serve as your database. In **Flows**, you can modify your data by merging data, adding new columns, and building dashboards. **Jobs** are the list of your completed data analytics tasks. It's like a folder with final reports. Use Connectors to connect to your datawarehouse and pull data directly from there. ### **Connect Data** Upon landing on the Tomat home page, the first step is to connect your data. Tomat offers two options for data connection: local CSV or Excel files and cloud database integration. If your data is stored locally, simply choose the local file upload option and select the desired CSV file. Alternatively, if your data resides in a cloud database, Tomat allows you to connect directly to it. Upon proceeding, you'll be directed to the exploration page. Here, Tomat provides a comprehensive set of tools to validate and control data quality, as well as examine value distributions for each column. By leveraging the exploration bar, you can gain quick insights into your data. For instance, let's say you want to filter out only active advertising campaigns. Tomat allows you to effortlessly apply this filter and analyze the results. ![](/files/MwijsCal3xUwhrz5XKK5) If your analysis concludes at this stage, you have the option to download the newly refined dataset directly to your computer. However, if you wish to further analyze or transform the data, simply click on "Add to Flow" to seamlessly continue your data preparation work. ### **Analyze data in** Tomat Now we are ready to start our data analysis project in Tomat! Once you have launched a data flow with your dataset, you will find a preview of it at the bottom of the screen. This allows you to have control over the data you'll be working with. On the right side of the screen, you'll notice a range of auto-cleanup suggestions. These suggestions can help streamline your dataset by removing empty rows or formatting text case, among other options. You have the flexibility to apply these suggestions or ignore them, depending on your specific needs and data requirements. Also, on the right sidebar, you will perform all your data transformations. At the top menu of the Tomat interface, you'll come across a range of operators referred to as Nodes. These Nodes serve as the building blocks for manipulating your data. They are similar in concept to what you might encounter when working with spreadsheets or SQL. These powerful operators enable you to carry out a wide array of data transformations and analysis tasks.
As an example, let’s start with creating a visual dashboard out of our dataset. To begin chart creation in Tomat, select the Chart node and specify the axes that will represent your data. This intuitive process allows you to choose the most relevant variables to visualize. Upon completion, Tomat generates a dashboard displaying the chart based on your specifications. ### **Enhance Analysis with Custom Metric** Tomat offers the flexibility to enhance your analysis by incorporating custom metrics. Let's take a step back in our analysis and introduce a new custom metric to our filtered table. By adding a new column, we can perform calculations that provide deeper insights into our data. To begin, click on the "New Colum” node on the Tomat menu. This action will prompt the appearance of the right sidebar, where you can define the formula or expression for the custom metric. Tomat offers a wide array of expressions that can be utilized to calculate the desired custom metric. ### **Download Your Results** To finalize the data manipulations, you can easily export your results by clicking on the “Output” node. From there, you have the option to select a destination folder on your computer or directly push the table to a cloud data warehouse. To apply all the transformations created, simply click on the "Run" button located at the top of the screen. Once the transformations are applied, the results can be found in the Jobs section. There, users will discover two insightful charts along with the transformed table, providing a comprehensive and visually appealing overview of the data analysis. ![](/files/Sul9A5HhJ4dcEY9z8O9N) Tomat's user-friendly interface and powerful capabilities make it an invaluable tool for any data analyst or researcher. Experience the power of Tomat today and unlock a wealth of knowledge hidden within your data. # FAQ {% hint style="info" %} Can't find the correct answer please ask our community in Slack [#general\_chat](https://join.slack.com/t/tabulaio/shared_invite/zt-2cugjcmpx-IDV_U4ga8mQ26J3W3kT6_g) channel. {% endhint %} ### APP USAGE
What’s the difference between flows and jobs? The flow is a streamlined process that transforms your data, beginning with the "Source" node and ending with the "Output" node. This is similar to a file in traditional applications. You can create new flows, rename them, make duplicates, export them, and delete them. Each table displayed within the flow is simply a preview of the final result and is not saved. A job is a process of executing the entire flow and saving the output in a file.
Which data transformation features does Tomat have? We cover all SQL features and some more such as auto-flattening JSON.
How do I run the job? When you have completed designing the flow and wish to save the result, click on the "Run" button located in the top right corner.
What is going on when I press Run button? The computation process will begin, utilizing either cloud resources or your local computer, depending on the destination specified in the "Output" node.
Where does Tomat execute my flow? The choice of resources for flow execution depends on the configuration of the "Source" and "Output" nodes. If Snowflake or PostgreSQL is connected, Tomat will execute the flow using their resources. If "Local File" is selected, execution will take place on your computer. To ensure proper job execution, the Tomat app must be running, and the computer should not be in sleep mode.
How does Tomat process data, does it push any data to the server? No, the application only verifies your credentials during the initial launch. Therefore, it is safe to use sensitive information in your flows as all data is stored on your local computer. We are actively working on implementing cloud features.
Does Tomat download my data from a warehouse or DB? Yes, the app downloads a small sample of data to generate a table preview in design mode. This sample data is stored on your local computer where Tomat is running. All the transformations execute utilizing warehouse or DB resources.
I have a large .CSV file that I cannot open with Excel. Could Tomat open it? Yes, Tomat can open it no matter the size! The default sampling is 10,000 rows, but you can change it on the Explorer page
Can Tomat work with BigData, large files, or tables? Yes, handling large databases is one of our main priorities. If you encounter unique problems when working with large databases, please report this issue in the [#general\_chat](https://join.slack.com/t/tabulaio/shared_invite/zt-2cugjcmpx-IDV_U4ga8mQ26J3W3kT6_g) channel.
Could I create a custom function to reuse in the future? Yes, we currently support scalar user-defined functions. To create them, please follow these instructions. It is important to note that any changes should only be made if you are fully aware of what you are doing. Remember to only include functions that are already working in the `body`. You can find custom-created functions in the directory provided. They are provided as examples. Open the \*.json file in your preferred text editor (such as Notepad++ or Sublime), modify the parameters, and save with a new name. Restart Retable and use them. * **For Mac OS:** “/Users/%username%/Library/Application Support/tomat/latest/udfs” * **For Windows:** “C:\Users\\%username%\AppData\Roaming\tomat\latest\udfs” *Where "`%username%`" is the name of the user.*
### INTEGRATIONS
Can I connect {sourcename} to Tomat? The application is currently in closed beta and undergoing heavy development. At this time, we have connectors for PostgreSQL and Snowflake. You can export the necessary tables from your database in \*.csv format and import them into Tomat. MS Excel \*.xslx format is also supported. It's important to note that \*.xslx files will only work if there are no Snowflake/PostgreSQL sources in the current flow. We plan to provide more connectors in future releases. To stay updated on product developments and share feedback on which integrations you need the most, join our [Slack Channel](https://join.slack.com/t/tabulaio/shared_invite/zt-2cugjcmpx-IDV_U4ga8mQ26J3W3kT6_g) for Data Analysts and Engineers.
How many data sources can I add? Theoretically, unlimited sources can be added. To be more precise, you can add up to 1024 different Source nodes.
Which sources or target connectors does Tomat have? We provide full support for PostgreSQL and Snowflake connectors as sources, targets, and for executing flows. Additionally, we support CSV and \*.xslx as a source and target.
### TROUBLESHOOTING
The application does not run on Windows For smooth functioning of our application on Windows OS, the Microsoft Visual C++ Redistributable library needs to be present in your system. Unfortunately, this library may be missing in some systems causing a disruption to the use of our product. Here is a simple step-by-step guide on how you can rectify this: 1. Visit the Microsoft official website or click [here](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170) to directly navigate to the download page of Microsoft Visual C++ Redistributable. 2. Initiate the download process for the redistributable library that corresponds to your particular version of Windows OS. 3. Once downloaded, install the Microsoft Visual C++ Redistributable library by following the on-screen instructions. 4. After successfully installing the library, restart your system to complete the installation process. 5. Now, continue with the installation or running of the Tomat app. Please note that this issue doesn't affect Mac OS users and thus these instructions are only applicable to Windows users.
Unknown authentication error during login If you're encountering an Unknown Authentication Error, it could be due to incomplete authentication steps. After you've installed and opened the app, you should see a 'Sign In' button. Once clicked, a browser window will appear with a login page. Depending on your method of registration, use the Email and Password fields if you've set a password from the registration email link. Conversely, if registered via Google mail, use the Google account login button. Upon successful signing in, a popup should appear - it is crucial to click on 'Open Tomat' to access the app's home page. Avoid switching the authorization window or delaying to press the 'Open Tomat' button because the authorization token has a limited lifespan. Delays may lead to the Unknown Authentication Error. In case you're unable to move past the Unknown Authentication Error screen, simply refresh the page by selecting **View** -> **Reload** or restart the app. Then, repeat the authorization steps.
Where can I find local logs to attach with my feedback? The log files a hosted below. If an error occurs during a job execution you can find logs on the job page `Three dot button → Show logs`. Locating the folder where log files are stored can be a challenging task, particularly if the option to hide system files and folders is enabled in the operating system's settings. **For Mac OS:** 1. Open any folder in Finder. 2. Press `Command + Shift + Period`. 3. See the hidden files appear in the folder. 4. Navigate to “`/Users/%username%/Library/Application Support/tomat/latest/logs`” For Windows 10: 1. Open File Explorer from the taskbar. 2. Select `View → Options → Change folder and search options`. 3. Select the `View` tab and, in `Advanced settings`, select `Show hidden files, folders, and drives` and OK. 4. Navigate to “`C:\Users\%username%\AppData\Roaming\tomat\latest\logs`” **For Windows 11:** 1. Open File Explorer from the taskbar. 2. Select `View → Show → Hidden items`. 3. Navigate to “`C:\Users\%username%\AppData\Roaming\tomat\latest\logs`” Where "`%username%`" is your name of the user.
{% hint style="info" %} Can't find the correct answer please ask our community in Slack [#general\_chat](https://join.slack.com/t/tabulaio/shared_invite/zt-2cugjcmpx-IDV_U4ga8mQ26J3W3kT6_g) channel. {% endhint %} # Home Page The Home Page is designed to offer a simplified yet powerful user experience. Here’s what you can expect. **Open your data file.** Clicking this button will prompt you to upload a data file directly from your computer. This is especially helpful for those who are just starting and may not have any existing data connections. **Create a data flow.** This button will navigate you to Flow Designer, where you can begin creating your data flow. If no data connections are created, this button changes to **Connect to Your Source**, guiding you to set up your initial data connections. #### Recent Datasets and Flows You’ll find lists of your recent datasets and flows. This allows you to quickly: * Resume work on existing data flows * Access datasets you’ve recently worked on * Stay updated on recent activities #### Navigation Buttons **Catalog**: Clicking this button will take you to the Data Catalog page, listing all your datasets. It's the library of all the data sources you can use for your flows. **Flows**: This button navigates to the Flows Page, which provides a comprehensive list of all your existing data flows. # Exploring Data ### [**Data Catalog**](/product-overview/exploring-data/data-catalog) The Data Catalog is the starting point of your data exploration journey in Tomat. It's where data becomes accessible and manageable. **Centralized data access.** You begin by accessing a centralized repository of all your datasets, ensuring a smooth start to the data exploration process. **Adding and organizing data.** The ability to add new datasets and organize existing ones empowers you to maintain a dynamic and up-to-date data environment. **Initial data overview.** This stage provides a preliminary understanding of the available data, allowing deeper exploration. ### [**Exploring Datasets**](/product-overview/exploring-data/exploring-datasets) Once on the Exploring Dataset page, you embark on a deeper exploration of their data. This is where detailed interaction with the data occurs. **Interactive data manipulation.** Tools for filtering, grouping, and sorting allow you to interact with data hands-on, tailoring the dataset to their specific needs. ### [**Statistics Panel**](/product-overview/exploring-data/statistics-panel) The Statistics Panel is a dedicated space on the right side of the interface that presents a statistical summary of each column in a dataset. **Statistical summaries for insight.** By offering key statistical measures for each column, users can quickly gauge the nature and distribution of their data. **Real-time statistical feedback.** As data is manipulated, the panel updates, providing immediate feedback on how changes impact the data's statistical properties. **Data-driven decision making.** This continuous stream of statistical information aids users in making data-driven decisions based on solid statistical foundations. # Data Catalog ## Overview The Data Catalog Page is a repository for datasets within the platform. It lists datasets along with pertinent metadata for each item. The page facilitates dataset management tasks, including previewing, quick navigation between datasets, and transitioning to the Explorer Page for detailed data analysis. ### Adding a new dataset Add a new dataset to the list with the **"+ Add dataset"** button. Once you select a file, you will see a preview of 100 first rows and settings similar to those in [Source Node](/data-transformation/transforms/source). If you are good with the dataset preview, click **"Add to catalog,"** the file will be opened on a full-screen in Explorer mode. You can then return to the Catalog page. ### Dataset list The list consists of columns providing the following information for each dataset: * Dataset name * Location. The path to the original file. * Type. File format. * Added. The date on which the dataset was added to the catalog. From the **context menu**, you can (three dots icon): * Open in data set on the Explorer Page * Go to file navigates you to the local file location on your laptop * Remove the file from the catalog (the original file won't be removed). #### Dataset Preview Clicking on a row corresponding to a dataset triggers a preview that will appear in the lower section of the screen. This preview allows you to inspect a dataset's contents without navigating to a different page. #### Quick Navigation The Data Catalog Page is configured to allow rapid navigation between dataset previews. This is implemented to assist in tasks that require the comparison of datasets or locating specific data entries. #### Explorer Mode An option is available to open datasets in Explorer Mode for in-depth data analysis. Explorer Mode provides a more expansive set of tools for data manipulation and examination. # Exploring Datasets ## Overview The Data Explorer Page is a dedicated page for manipulating a single dataset. It offers tools housed in a toolbar for dataset manipulation and customization, including grouping, filtering, sorting, and more capabilities. Additionally, it provides a Stats Panel for detailed statistics by columns. This documentation covers the features and usage of the Data Explorer Page. By default, the page displays a sample of 10,000 rows. You can **change the sample size** and select the sample type, either 'Use the first rows' or 'Use a random set of rows.' Click the "setting" icon next to the dataset name to call a popup with the settings. ### Toolbar Options The toolbar at the top of the Data Explorer Page comprises various options for data manipulation: **Group by.** Allows you to group rows based on selected columns. **Filter.** It provides the capability to apply conditional filters to the dataset. **Sort.** Facilitates ordering of rows based on selected column criteria. **Hide.** Enables hiding of selected columns from the view. **Data Bars.** Offers a graphical representation of data for easy comparison. #### Convert to Flow Button The 'Convert to Flow' button lets you convert the current dataset view into a flow within the Flow Page. It also applies all current filters as transformation steps automatically. #### Save As Button Use the 'Save as' button to apply current filters to the entire dataset and save it as a CSV file. It can take a while as filters are applied to the whole dataset, depending on the dataset size and your computer engine. ### Stats Panel On the right-hand side of the page, a Stats Panel is available, offering column-wise statistics such as mean, median, standard deviation, etc. See [detailed documentation](/product-overview/exploring-data/statistics-panel) on how to use column statistics. #### #### # Statistics Panel Exploratory Data Analysis ## **Overview** Understanding your dataset is crucial before undertaking any analysis. The Statistics Panel in Tomat offers a range of profiling widgets to make this process straightforward and insightful. ### **Getting Started** 1. Open your dataset in the Explorer Page - you will see the Statistics panel on the right side next to the table. 2. Alternatively, open the Flow Page, select any node, and select the "Stats" tab in the right panel. Depending on the columns you select, you will see different widgets and visualizations: **No columns are selected.** A summary widget for each column is displayed. Click on a row above the widget to see extended analytics for a specific column. **One column is selected.** A set of widgets related to the selected column is shown. **Two or more columns are selected.** A list of summary widgets for the selected columns is displayed. You can drill down into any widget for more information. ### **Profiling Widgets** The following widgets are available for data profiling: **Data Quality Widget.** Displays the summary of data quality for all rows in a column. **Summary Widget.** Shows summary statistics based on the column's data type. Contains main and additional statistics. **Frequency (unique values) Widget.** Displays the most (or least) frequent values in a column, sorted by default in descending order. **Histogram Widget.** Visualizes the distribution of values in columns as bars. **Periodic Histogram Widget.** Available for periodic data types only (e.g., DateTime, Date, Time). Shows value distribution grouped by parts of data, such as year, month, week, day, and hours. #### **Data Types and Widgets** Depending on the data type of the columns, specific widgets, and visualizations will be displayed: * String: Data Quality, Summary, and Frequency * Boolean: Data Quality and Frequency * Integer and Decimal: Histogram, Data Quality, Summary, and Frequency * DateTime, Date, and Time: Histogram, Periodic Histogram, Data Quality, Summary, and Frequency * Array and Object: Data Quality # Designing Flows ## Overview Visual data flows are a graphical representation of data transformation processes using a series of interconnected nodes. Each node in the pipeline represents a specific data transformation action or operation, such as filtering, sorting, or aggregating data. The purpose of visual data pipelines is to provide a user-friendly, intuitive way to design, configure, and understand complex data processing tasks without the need for extensive programming knowledge. #### Key aspects of visual data pipelines **Nodes.** Nodes are the fundamental building blocks of a visual data pipeline. Each node serves a specific purpose and encapsulates a particular data transformation operation, such as filtering rows based on specific conditions, sorting columns, or adding new columns using expressions. **Connections.** In a visual data pipeline, nodes are connected to define the data flow through the transformation process. The output of one node becomes the input of the next node in the sequence, allowing you to chain together multiple transformations to achieve the desired result. **Visual Interface.** The primary advantage of a visual data pipeline is its user-friendly graphical interface. You can add nodes on a canvas with one click, connect them in the desired order, and configure the settings for each node using a property grid. **Real-time Preview.** Our data pipelines include a real-time preview feature that allows you to see the impact of their transformations on a sample of the dataset instantly. This helps you understand the results of their configuration choices and adjust settings as necessary. # Creating Flows ## Overview You will design and set up your data transformations on the Flow page. A data flow consists of connected nodes that perform different transformations to process data in Flow Designer. When you build a data flow, you add and connect nodes. You also configure those nodes and workflow properties. To make a new flow, go to Flows Page and select **Create a Flow,** or go to Home Page and select **Create a Data Flow.** Flow connections move in a downstream direction horizontally. In this mode, you will work with sample data to apply and preview results in real-time. ## How It works 1. Open the app and select "Create" on the Flows pages 2. **Add Transform Nodes**. In Design Flow mode, various nodes are available, each performing specific tasks, such as sorting, filtering, or merging data. To include a node in the flow, click on it in the toolbar, and it will be added to the canvas. 1. As the first step, you need to add a Source node and select your source dataset. You can add any number of sources you need. 3. **Connect Nodes.** If any canvas nodes are selected, a new node is automatically set as an input connector. If not no nodes were selected, you would see a blue frame around the canvas; you need to click on the node you want to connect to. 4. **Configure Draft Nodes.** Each node features settings that can be adjusted to customize the transformation. Click on a node to open its settings panel and modify the necessary parameters. 5. **Realtime Preview.** While configuring the node’s parameters, you will see a result table preview in real time. 6. **Apply the Transform**. Press the Apply button on the settings panel once you are good with the transformation result. 7. **Add Destination**. Add node Output and set your destination dataset where the result should be saved. You can add more than one destination. 8. **Execute Flow**. Press the "Run" button to execute the flow. # Flow Designer Guide # Working with Canvas ## Basics of Canvas **Add and Edit Nodes** Select a node on a toolbar and click on it; the node will be added to the flow and automatically connect to the selected node on canvas. Click the right mouse button to open the context menu, where you can: * rename the node * disconnect the node from the previous one in the flow * duplicate * delete #### **Connect Nodes** If no nodes were previously selected on the canvas and a new node was added, you will see a blue frame around the canvas and sign "Select input node." You need to select a node to connect to and click on it; a new node will be connected to the selected one. You can disconnect the node via the context menu. #### **Multi-node selection** You can select multiple nodes, see [instructions](/product-overview/designing-flows/flow-designer-guide/working-with-canvas#multi-nodes-selection), and then: * drag-and-drop selected nodes on the canvas * delete all selected nodes via the context menu * group nodes, see [instruction](/product-overview/designing-flows/flow-designer-guide/using-groups) * add the union transform to merge selected datasets ## Canvas Controls #### Multi-nodes selection * **Mouse:** Shift + Left button - hold and drag * **Trackpad:** Shift + Imitation of the left button - hold and drag #### Zoom in and out * **Mouse:** Hold down ⌘ Command (Mac) or Ctrl (Windows) and scroll the mouse wheel up to zoom in or down to zoom out. * **Trackpad:** Pinch two fingers together to zoom out or stretch two fingers apart to zoom in * **Hotkeys:** * Zoom in: Shift + * Zoom out: Shift - * Zoom to fit: Shift 1 ## **Canvas Menu** #### **Centralize** Use the centralize button on the lower right corner of the canvas to reset zoom and canvas position. #### **Minimap** Hover over the centralize button and wait till the menu is opened; select a minimap to see the whole flow at a glance. #### Text Note Hover over the centralize button and wait until the menu opens; select a note. You can change a note's color text and move it around the canvas. Open the context menu on the right mouse button to remove or duplicate the note. # Using Groups ## **Adding Groups** You can select multiple nodes, see [instructions](/product-overview/designing-flows/flow-designer-guide/working-with-canvas#multi-nodes-selection), then press **G**, use the context menu, or select an action **Group Nodes** from the right panel. A new group will be created. Collapse the group by clicking on the arrow icon in the header. Open back by clicking on arrows over the left upper group corner. ## **Editing and Removing Groups** **Rename**. In the right panel, select the Transform tab to rename the group and see how many nodes it contains. **Add new nodes**. Drag the node over the group, the group canvas highlighted in blue, then drop the node, and it will be added to the group. **Ungroup**. Select "Ungroup" in the context menu. Use it to remove grouping without removing nodes inside. **Duplicate group**. Select "Duplicate" in the context menu. The group will be duplicated with all the nodes inside. **Remove group.** Warning! group will be deleted within all the nodes inside it. Select "Delete" from the context menu. **Change the background color**. Click on the grey circle over the group header and select a new color from the palette. # Working with Table Tables are a foundational element in data management and analytics, and Tomat offers a versatile table interface that allows for multiple functionalities, making your data handling smoother and more efficient. This article will discuss the key features of tables in Tomat, including their Preview and Simple modes, how to interact with them, and much more. ## Table Modes ### Preview Mode When you are editing the node, the table is in Preview Mode. This mode is designed to review data and assess how your changes affect the dataset. Any changes you make to the data will be *highlighted*, providing immediate visual feedback. There are different types of highlighting: * new column(s) is added - green background * the existing column will be removed - red background * the existing column will be replaced with a new one - you will see both the old (grey background and red icon in the header) and new column (green background) alongside * the existing row will be removed - red background * the existing column will not change but is involved in the current transform settings - grey background Use the toggle "**Show only the affected columns**" in the footer to hide all columns not involved in the current transformation. ### Default Mode After clicking "Apply" on your transformation, the table will shift to Default Mode. This mode allows more direct interaction with your table data. #### Edit in Place This feature streamlines the data modification process by allowing you to perform several tasks directly on the table. * Rename columns. To rename a column, all you have to do is double-click on the column header. This action will transform the header into an editable text field. * Move columns. You can change the columns' order by dragging the column header and dropping it at the desired location. * Change column type. You can also modify the data type of a column directly from the table view. Click on the data type icon next to the column name to change and select a new type from the menu. #### Context Menu The context menu offers a list of options to manipulate columns. This menu is accessible by clicking the icon with three dots next to the column name when the table is in Simple Mode. * Rename. Selecting "Rename" opens up an editable text field for the column name. * Insert Left and Insert Right. These options allow you to insert a new column to the selected column's left or right. * Sort A-Z and Sort Z\_A. Options to sort the column data in ascending or descending order. * Move to Beginning, Move to End. Options to move the selected column to either the table's beginning or end, respectively. * Hide and Delete. The "Hide" option will make the column invisible but won't remove it from the table, while "Delete" will remove the column. You can **download** the table as a CSV file or copy its contents from the menu in the footer. Only sample data used in Flow Designer will be downloaded. ## General Features #### Columns Search Panel This panel appears on the left side of the table view, allowing you to search for columns and quickly navigate to them. #### Full-Screen View You can open the table in a full-screen view for a more immersive data exploration experience. Drag and drop the upper border of the table header till the table sticks to the toolbar. #### Long Text Cells Cells containing long text can be expanded in a popup window where you can have a clear view of your data and navigate between rows. #### Convert the row to the header. Select a row by clicking on it's number. You can convert the selected row into a header row. #### Filter Bar This bar contains options like filter, group by, sort, and bars. It's intended for quick data exploration and doesn't create new transformations in the flow. * Filter. This option lets you narrow down the data in your table based on specific conditions. You can select a column and specify the criteria for the data you want to view. * Group by. This function aggregates your data based on one or more columns. It essentially combines rows that have the same values in the specified columns. * Sort. Sorting allows you to arrange the rows in your table based on the values in one or more columns. You can sort data in ascending or descending order. * Data Bars. The "Bars" feature visually represents the values in a column with horizontal bars, similar to a mini bar chart within each cell. * Show formatting marks. It reveals hidden characters like spaces, tabs, and line breaks within the table's cells when activated. * Hide columns. The "Hide Columns" option makes specific columns invisible in the table view. This is different from deleting a column; hidden columns can be easily brought back into view. #### Column Header Statistics You can find quick column statistics right in the column header. The range of values is displayed for numerical columns, whereas the number of unique entries is shown for categorical columns. A colored line beneath this statistical information visually indicates the column's ratio of valid to missing data. Hovering over this line will reveal the numerical details of this ratio. Clicking on statistics information will expand it to show additional statistics like data distribution or the top unique values in that column. #### # Managing Flows Numerous actions are at your disposal for effective flow management. We'll delve into each of them below: 1. Changing your flow's name 2. Removing your flow 3. Creating a duplicate of your flow 4. Export flow 5. Import flow ### Renaming your flow To change the name of a Flow, you have a couple of options: 1. Click on the current name of your Flow, and you'll be able to edit it directly. 2. Navigate to your Flows page, open the context menu for the specific Flow, choose the "Rename" option, and then type the new name into the "Flow Name" field. ### Deleting your flow You have two methods to remove a flow: You can either delete it while in the flow designer, accessible through the navigation bar menu, or from your Flows page by utilizing the context menu. ### Duplicating your flow You have two options for making a copy of an entire flow: You can do it either from inside the flow designer through the navigation bar menu or from your Flows page via the context menu. This action will result in a duplicate of your flow, preserving its current location. # Demo: Building a Simple Flow This is a step-by-step tutorial on how to launch your first project in Tomat. Let's get started! **Part 1. Connect Data.** From the home page, you can connect your data. You can either use local CSV or Excel files, or connect directly to your cloud database if you use one. In this tutorial, we will work with local datasets. Before we begin, let's go through the Tomat interface. In our Catalog, you can store files that you work with. Remember that Tomat is a desktop app, which means no data will leave your computer and appear somewhere in the cloud. We prioritize security the most. If you do not have a data warehouse, our Catalog will serve as your database. In Flows, you can modify your data by merging data, adding new columns, and building dashboards. Jobs are the list of your completed data analytics tasks. It's like a folder with final reports. Let's go back to the home page and start working with data. I select a local file upload, pick a CSV, and preview it. Then I click on "Add to Catalog" to keep it handy for future use. The next window you see is our exploration page. First of all, you can check data validity and control data quality and value distribution for each column. Using the exploration bar, you can get quick insights into your data. For example, let's filter out only active advertising campaigns. If you're done with your analysis at this step, you can already download this new dataset onto your computer. To continue working with the data, click on "Add to Flow." **Part 2. Analyze data in** Tomat Now we are ready to start our data analysis project in Tomat! As you can see, we have uploaded a file with Facebook campaigns. At the bottom, you can preview your dataset. On the right side, you see auto-cleanup suggestions, such as deleting empty rows or formatting the text case. You can apply these suggestions or simply ignore them if you don't need them. Also, on the right side, you will perform all your data transformations. As you remember, we have filtered our initial dataset, so the operation is recorded here in the data pipeline. In the top menu, you see different operators that we call Nodes. Use them to manipulate your data. They are similar to what you have in spreadsheets or when working with SQL. Now, let's build a chart. I select a Chart node and specify the axes. The dashboard is created. In our future releases, it will be possible to download not only a static image but also a dynamic dashboard with connected data. Let's take a step back and add a new custom metric to our filtered table. To do this, I will add a new column and, as an example, divide the budget by the number of clicks to calculate the CTR (Click-Through Rate). And I can create a new chart with the calculated metric. **Part 3: GPT Magic Node** Now let's explore how the magic AI feature works in Tomat. You can format data or create new values using natural language. For example, I can ask the AI to perform a new calculation. Don't forget to click on the purple button to launch the GPT command. Later, I will show you more advanced examples of how to use AI in your data analysis. **Part 4: Downloading Your Results** When you are finished with your data manipulations, click on the output node. Then, select a destination folder on your computer or push the table directly to the cloud data warehouse. After that, click on "Run" at the top of the screen to apply all the transformations you have created. Go to the jobs section and find your results: two charts and the table. **Part 5: GPT Tutorials** I suggest studying our tutorials available in Flows, particularly paying attention to the GPT tutorial. You can format unstructured data like phone numbers or translate text into another language. Explore our examples, and we look forward to seeing your own AI use cases. Don't hesitate to share them with our community in the Slack channel. Here is a quick overview of how you can handle data in Tomat. If you need more tutorials like this, please let us know! See you in Tomat! # Executing Flows ### [**Running Flows**](/product-overview/executing-flows/running-flows) This section describes how you can actively run your data flows. * How to run a flow * Execution of flow ### [**Jobs Overview**](/product-overview/executing-flows/jobs-overview) The Jobs Overview is a comprehensive dashboard that provides a detailed view of all scheduled and executed jobs. * List of all jobs * Scheduling a job * Viewing results # Running Flows ## Overview Once you have set up your flow in Design Flow mode, you can move on to the Runtime mode. In this mode, you will execute the transformation flow on your entire dataset by simply clicking the "Run" button. As your dataset can be large and dependent on the complexity of the flow you created, running the transformation over the whole data can take significant time. To execute flow: 1. Verify the connections to your data sources and that you have destinations (local files or data warehouses). 1. Flow cannot be executed without output nodes or charts 2. Press the "Run" button to execute the transformation flow on the entire dataset. We call it “Job” 3. Once a job is done you get a notification and can go to the job either from the Jobs page or from the Flow by pressing the Job button near the right corner of the toolbar. ## Where does Tomat execute transforms? {% hint style="info" %} Under the hood, Tomat generates SQL for each dialect agnostic transformation that could be executed on any warehouse or database. {% endhint %} In the Design mode, all transforms are executed locally using the engine and power of your laptop. No data is uploaded online. In Runtime Tomat supports two types of execution engines: local and data warehouse. The type of execution engine used for the flow depends on the data sources and destinations you have selected: * If all data sources and destinations are local files, the flow will be executed locally like in design time, but using the whole datasets. * If any data source or destination is a table in a data warehouse, the flow will be executed in that warehouse. In this case, source local files will be temporarily uploaded to the warehouse and removed after the flow execution is completed. If there are output local files then they will be downloaded and saved on your laptop. At the moment, Tomat chooses the execution engine automatically. But we are going to support a manual selection of the engine in the near future. # Jobs overview ### Introduction The job page provides a comprehensive view of each job, including scheduling options, a list of outputs from the last run, and a detailed history of all runs. ### Accessing the job page #### From the data flow designer In the data flow designer, once your data flow is ready, click the **Run** button. This action will either create a new job if it doesn't exist or initiate a run instance for an existing job. When job finishes working, you will see a notification and can access the job's result directly from the notification or by clicking the **Job** button on the toolbar. #### Direct access You can access existing jobs by navigating to the **Jobs** section from the Home Screen. Note: You cannot create a new job directly from the Job page. Jobs are only created through the data flow designer. ### Understanding the job page layout #### Scheduling Customize when and how often your job runs. This section allows you to configure recurring schedules or one-time executions. #### Outputs of the last run This area displays the results from the most recent execution of the job. Outputs can include reports, charts, or datasets. Click on an output to get a quick preview. #### Run monitor Here, you see a chronological list of every instance the job has run. You can drill down for each run to view detailed information such as status, outputs, and logs. ### User flow for monitoring a job **Start a job.** In the data flow designer, click the **Run** button to initiate a job or a new run of an existing job. **Job creation.** If the job doesn't exist, it will be automatically created. **Monitoring the job.** Once the job starts, you can go to the Job page. * View the **status** of the current run. * Check the **outputs** from the last run. * Look through the **history** of all runs. ### Tips for effective job management **Regularly check schedules.** Ensure your jobs are running at the intended times. **Preview outputs frequently.** Regular previews help in catching errors or anomalies early. **Review logs.** In case of failures or unexpected results, logs are invaluable for diagnosing issues. **Update data flows as needed.** Changes in data flows are reflected in subsequent job runs. Keep your flows updated for accurate results. # Building Reports ### [**Designing Reports**](/product-overview/building-reports/designing-reports) This creative hub is where users bring their data to life through reports. **Interactive report creation.** You can start designing your reports by selecting data flows and utilizing a variety of elements like tables, charts, text blocks, headers, and dividers to create a structured and informative report. **Customization and flexibility.** The design interface offers significant customization, allowing you to arrange different components to match their reporting needs. **User-friendly interface.** The process is supported by a user-friendly interface, making it accessible even for those with minimal design experience. ### [**Running Reports**](/product-overview/building-reports/running-reports) Once the design is set, reports are generated by executing data flows. **Automatic generation.** When a data flow is executed, the corresponding report is automatically generated, ensuring the content is always synchronized with the latest data. **Viewing and access.** Generated reports can be accessed on the Job page or the Reports page, providing a centralized location for users to review and analyze their data. **Static nature of reports.** The reports are presented in a non-editable format to maintain data integrity, with any necessary edits being made by the report designer. # Designing Reports ### I**ntroduction** In Tomat, designing reports is a seamless and intuitive process, closely integrated with data flows. Reports in Tomat are directly linked to data flows on a one-to-one basis, providing a dynamic way to visualize and present the results of your data manipulations. ### **Creating a New Report.** 1. Starting from a data flow. To design a new report, open the data flow you wish to visualize. 2. Switch to the report tab. Within the data flow interface, switch to the 'Report' tab to start designing your report. ### **Adding Blocks to Your Report** **Source and output nodes (Tables).** Incorporate tables from your data flow into the report. These can be either source nodes or output nodes, depending on what part of the data you wish to display. **Chart nodes.** Add chart nodes to represent data insights visually. These charts are directly linked to the data within your flow for accurate and dynamic data visualization. **Adding blocks.** Blocks can be added either through the context menu in the flow or by selecting them on the report page. ### **Customizing the Report Page** **Moving Blocks.** Users can rearrange blocks on the report page, allowing for a customized layout that best fits the report's narrative. **Adding New Blocks.** Enhance your report with various block types: * Table blocks. Add tables that represent either source or output nodes from your data flow. * Chart blocks. Incorporate charts for graphical data representation. * Metric block. Add source or output tables as a source of metrics. Metric always shows only one value - the first row from the first column. The name of the metric is the column name. * Header blocks. Use headers for titles and section breaks, providing structure to your report. * Text blocks. Include free text for annotations, descriptions, or analysis. * Divider blocks. Use dividers to separate different sections of the report for better readability visually. **Navigation Panel.** On the left side, a navigation panel similar to Google Slides is available. This feature lets you quickly navigate and moving different blocks, enhancing the ease of report editing and review. # Running Reports ### **How Reports are Generated** **Automatic report generation.** When you run a data flow, the corresponding report is automatically generated. This ensures that every report reflects the most current state of the data flow it's associated with. **Viewing reports.** Generated reports can be viewed on the Job page or the Reports page. This allows easy access and review of the reports generated from executed data flows. ### **Content of Reports** **Data representation.** Reports display information directly from the real source or output nodes of the data flow. This means the data you see in the report is the most current and comprehensive view of your entire dataset. **Charts in reports.** Charts included in the reports reflect the results over the whole dataset. They provide a visual representation of the data, making it easier to understand trends, patterns, and insights. ### **Characteristics of Reports** **Non-editable format.** Once generated, reports are in a non-editable format. This ensures the integrity and consistency of the data presented in the reports. **Report designer for edits.** If changes are required, you should return to the [report designer](/product-overview/building-reports/designing-reports#introduction) to make edits. You can adjust the layout, content, and data visualizations to update the report. # Reports Page ## Overview The Reports Page is a repository for all reports within the platform. Here, you can preview reports, open them on full screen, and re-run (update) them. ### Adding new reports Reports appear on the Reports page when they are created inside data flows. Once you switch in a data flow from designer to a flow and create a first element a new report is created and appears on Report page in "Draft" status. To activate the report, you need to run the data flow, and the job should be successfully finished at least once time. Then, you will see your report in "Active" status on the Report page. ### Reports list The list consists of columns providing the following information for each report: * Report name * Updated date * Last run date * Status: "Draft" or "Active" From the **context menu**, you can (three dots icon): * Open in report * Edit. Go to the report designer. * Rename * Export as PDF * Export as HTML * Go to flow. Open flow designer. * Delete report. #### Report Preview Clicking on a row corresponding to a report triggers a preview that will appear in the lower section of the screen. This preview allows you to inspect a report's contents without navigating to a different page. #### Open Report To open the report on full screen, select "Open" from the context menu or click on the "Open" icon from the preview mode. # Tomat Use Cases As you embark on your journey with Tomat, it's essential to grasp the capabilities of this tool and how it can simplify your tasks. At its core, Tomat is designed to enhance productivity and streamline workflow. ### By departments #### Marketing Marketers can use Tomat to track campaign performance, customer engagement, and ROI. #### Operations The Operations team can monitor process efficiency, optimize workflows, and manage assets. #### Product Managers Product Managers do feature usage analysis, customer segmentation, and prioritizing product roadmaps based on data insights. #### Sales Department Sales teams can utilize Tomat for lead scoring, sales forecasting, and customer segmentation. #### Customer Support Customer support can analyze customer interactions, resolve issues faster, and improve customer satisfaction. #### Research & Development (R\&D) R\&D departments use Tomat to manage datasets, perform exploratory data analysis, and run machine learning experiments. #### Human Resources HR teams can benefit from tracking employee performance, managing recruitment processes, and improving workforce planning. ### By Industry #### Healthcare Analytics Healthcare organizations can use Tomat to analyze patient data to optimize treatments, predict patient outcomes, and improve operational efficiency. #### E-commerce E-commerce platforms can leverage Tomat to analyze customer behavior, optimize product recommendations, and forecast sales. #### Supply Chain Management Logistics and supply chain businesses can use Tomat to monitor inventory levels, predict demand, and optimize routes. #### Finance and Banking Financial institutions can use Tomat for fraud detection, risk assessment, and customer segmentation. #### Manufacturing Manufacturing companies can employ Tomat for quality control, predictive maintenance, and production optimization. # Transforms The concept of a node is central to the data transformation process. Each node represents a specific data transformation action with its associated settings. Nodes are the building blocks of the data ### Source and output transforms
SourceAdds outer dataset to the flow/files/9wsooLNGtZNR78T20sxr/pages/WgnHh23mscj6YZGTuE3K
OutputSaves the result of data flow in an outer dataset/files/O6AIfQ8kNNgL4i4ZtLiY/pages/0Hgl21Io7LQyZ9KOzfrD
Empty tableAdds empty table where users can manually add data/files/ddVSddTlqvXMszcDhINo/pages/mLOtLMY5Lv6CHpkqFqXs
ChartShow charts/files/CNekNhY6Rs5M110el1jk/pages/T3fllRuwGmXMkjUDTbPz
### Cleanup transforms
New ColumnAdds a new column with “text”, number, or function/files/0sz4k0ycH8zBYGWRHgPM/pages/qgCmJVrLNCWykOEj76R6
Change column typeChanges a data type for selected column(s)/files/N8IdqvwvLZRbIwwMkMiq/pages/GnqELalwD6jLe9AxYfdi
Columns EditRenames, deletes and moves columns/files/q2kPsO4pJ2l5BCmqK0UT/pages/Fnw1rECiTiGejppWJ7Co
If...ThenAdds a new column with a value based on the specified condition/files/Pc21BH1qoDVXxxHuIyFs/pages/lvf3SXBkAU0yQY65XdmZ
SortSorts a table by the specified column(s)/files/awJwCIgBV15GSjTjzuJC/pages/HKeNqCrNWfZF4cFXBRSN
FilterFilters rows based on the specified condition/files/Uyiu5MRkcuwOsNxK9gpk/pages/N6lqlo52oMZxxgFiMVCc
Remove DuplicatesRemoves duplicated rows/files/YkVRHZ0hHw71gxTsXMfg/pages/8uaLucUyJz4g4wlWLHY0
Split TextSplits a column with the specified delimeter and returns the result in the new columns/files/nDBB7b6KvCsM9YTaWWx5/pages/ERUhieEis5D0DdZJmK3x
Extract TextExtracts the specified part of text into a new column(s)/files/qLJJ1EcjdlfG36OCpyAS/pages/tjowAC0eeRxOckn9OjPS
Find and ReplaceFinds and replaces the specified part of text/files/qcZDV0m7ZoUx0ZW8oHxc/pages/JZc9XQCutE2p27AK2R2u
Match TextCounts matches based on specified pattern in a column(s)/files/kntBahsLALtg3sWtcE9T/pages/ZtlEWuMoepZryK9sjWeU
Rolling FunctionsCalculates a window function operates on a group (“window”) of related rows/files/Y0blER2D4ta5KUFFQPww/pages/3SoAz0W8ZxvXgh3sgdQ4
NestCreates Objects or Arrays in JSON format from the specified columns/files/KQKNCmKCEMOBvjH2u7Zj/pages/WdcUcPtnDPdhdXKiCOtM
UnnestFlats Objects of Arrays in JSON format into columns or rows/files/jiirwCy5e8mdqahfG2x0/pages/Shugd9s1AJ8E3flHAqSI
### Advanced transforms
JoinJoins 2 tables using the specified columns as keys/files/VcUDOFFTK69jAPKFr2ge/pages/I1pCeM12aDqupuck3DSv
UnionStacks rows of 2 or more tables/files/8HjRMwOCuqfV41wv85wE/pages/IUQQv3hzMjo5GtZ021iU
Group byGroups rows and computes aggregate functions for the resulting group/files/gZyJm5DmAqK2eYlJ5NoM/pages/xilWRhWIU0jht5r8iMpa
PivotCreates new columns from values in the specified columns and computes aggregate functions as values for the new columns/files/YK0yQhP8y49FmmcctYv7/pages/L7KQq6pYmDAezpnQRGYA
UnpivotReshapes the data by merging one or more columns into key and value columns/files/XhFdVSOiSE5iLb1LeDes/pages/oI3KMyIahB4iTUGNW4CM
### AI transforms
Magic ColumnAdds a new column based on GPT answer/files/qKUo0CZiZYAL3S4KD1lo/pages/WgnHh23mscj6YZGTuE3K
# Source Adds datasets to the flow ## Overview The Source Node is the starting point in your data transformation journey within Tomat. This crucial component allows you to seamlessly connect to various data sources, such as Snowflake and Postgres, or local files, such as CSV and Excel. You can easily import data sets into your workflow by simply dragging and dropping the Source Node onto your canvas. {% hint style="info" %} If you want to connect to cloud apps like Google Ads or Hibspot, please book a demo with our team. {% endhint %}
## Settings You can add local files from your computer or connect to databases and warehouses. ![](/files/1LXcGVaulsR037FPqTEd) ## Adding the local file If you want to add a file (CSV or Excel) from a local computer, select **Local file** and specify the path. When you select a file, you can preview 100 first rows at the bottom of the screen, and in the right panel, you'll have options to customize the upload. {% hint style="info" %} You can connect to a file of any size, as Tomat reads only sample (10,000 rows) data for design mode. During a job running, the whole file will be read. {% endhint %} **First row contains headers** If your file contains column headers - enable this option to convert the first line of the table to the headers. **Auto-detect data types** This option automatically checks each column's content and assigns the appropriate type (String, Integer, etc.) to the columns. If you do not select this option, all columns will be imported as Strings.
Advanced settings **Values separated by** This parameter specifies the character used to separate columns in the file. You can choose any of the most popular delimiters, specify your own, or leave the option of automatic detection. **Charset** Allows you to specify which charset is used in the file manually. **Quote symbol** In CSV and Excel files, a quote symbol (usually double quotes) encloses fields containing special characters like commas or line breaks. This ensures that the data within these quotes is treated as a single unit during parsing, maintaining accurate data structure. **Quote escape symbol** A quote escape symbol, usually a backslash (\\), is employed in CSV and Excel files to indicate that a quote enclosed within quotes should be treated as part of the data. This prevents misinterpretation of the quote as a delimiter. **Maximum columns count** This determines the highest number of columns allowed in a single row of a spreadsheet. It restricts the data structure by specifying the maximum number of separate data fields present side by side within a single row. **Maximum cell length** This option specifies the maximum number of characters a single cell within a spreadsheet can contain. This setting is crucial for ensuring data integrity and preventing issues from long text entries, which could impact file readability, system performance, or compatibility with other software.
## Connect to table or view in cloud warehouse. If you want to connect to a dataset from a database, first, you need to create a connector - this can be done by selecting the option **Add connection** from the Source connection list or in the main menu via the Connectors tab. **Database (only for Snowflake)** Select the database you want to connect to. **Schema** Select a scheme from the list. **Tables and Views** Specify which table or view we want to add as a data source. **Columns & Filter rows** Select which columns to add to the source or filter the rows in the original table. {% hint style="info" %} Don't forget to click the Create button to finalize the table import and Source node creation. We only collect a sample (10,000 rows) to increase performance and reduce costs. {% endhint %} ## Limitations At the moment all sources in one data flow should use the same connector to external tables. So you cannot use datasets from Postgres and Snowflake in the same flow, but you can combine local files with any type of external connectors. # New Empty Table Adds empty table to the flow, you can manually add and edit data. ## Overview The Empty Table Node offers a straightforward way to manually input data directly into your workflow. An empty table interface appears when you add this node onto your canvas, allowing you to type in your data or paste it from your clipboard. This feature is particularly useful for incorporating ad-hoc data, running quick tests, or supplementing existing data sets.
## Settings You have two options for creating a table. ![](/files/7NmiuvtzaURbqoIboupf) ## **Create manually** **Adding and removing rows and columns** To add and remove rows: * Use the Rows fields in the right panel to specify the desired number of rows. * Use the + icon after the last row in the table to add a new row. * Press "Enter" after editing any cell in the last existing row to add a new row. * Select "Insert row above" or below options in the context menu (right-click over a row number) * Select "Duplicate row" in the context menu * Select "Delete" in the context menu to remove the row. To add and remove columns: * Use the Columns fields in the right panel to specify the desired number of columns. * Use the "+ Add" and "X" buttons near the column list in the right panel to add or remove a new column, respectively. * Use the + icon after the last column header in the table to add a new column. * Select Insert left or right options in the context menu (right-click a column header) * Select "Delete" in the context menu to remove the column. **Rename and set type for columns.** {% hint style="info" %} Now we support a restricted number of types: String, Integer, Decimal, Boolean, Date, DateTime, Time. {% endhint %} * Use the right panel to set names and types * Double-click on the column name in the table header to rename the in-place * Click on the data type icon in the column header to change the type **Editing cells** Use arrows to navigate between cells. Fill with values depending on column type: String -> fill cells with free text; no quotes are needed. Decimal -> use a period (.) as a decimal part separator Boolean -> use values *true* or *false* DateTime -> use format yyyy-MM-dd HH:mm:ss (ex. 2023-04-25 13:54:21) Date -> use format yyyy-MM-dd (ex. 2023-04-25) Time -> use format HH:mm:ss (ex. 13:54:21) {% hint style="info" %} You can copy an existing table from Excel or CSV (using hotkeys Ctr+C or Cmd+C, depending on your operating system) and paste it directly into the table in the app by putting the focus in the first field and pressing Ctrl+V or Cmd+V. {% endhint %} ## Ask AI Set a prompt, and AI will automatically generate the table. The AI creativity slider sets the "freedom" for AI to generate the result, whether it should strictly match the query or allow deviations. To get the result from AI, click on the airplane icon first, and if you are happy with the answer, hit "Create table". Hit the "Start new chat" button to start a new chat and forget the previous context. # Output Saves the result of data flow in an outer dataset. ## Overview The Output Node is the final step in your data transformation process. It allows you to materialize the results of your data flows directly into your database platforms like Snowflake and Postgres or save them as a local file.
## Settings With the Output node, you can materialize results as a table or a view directly into Snowflake or Postgres or save them as a file. You only could select connectors that were used in Sources, or if you didn't use any external connectors in Source nodes, you are free to select any. ![](/files/xdFfN5aXkWUfyVKSwwXY) ## Saving to a local file ![](/files/3opVvcLHgjxAKjPUZbUV) **Destination folder** Click on the folder icon to open a window for selecting a folder on the local computer where you want to save the file. **File name** You can choose between two file types to save and specify a name. * \*.csv - comma-separated values * \*.xlsx - classic MS Excel format **Save columns' names as the first row** If you don't want to save table headers, turn this off. By default, the option is set to save column names as the first row. **Save options** ![](/files/3SaogpiLe9Fue1RwOsi4) **Rewrite the existing file**. Delete the file if it exists and create a new one on each run. **Create a new file.** Create a new file on each run with a timestamp suffix added to the name. **Append**. Append new data to the existing file on every run. If a file does not exist, create it.
Advanced settings In the advanced settings, you can select which character to separate values: comma, semicolon, tab, space, or custom symbol.
## Materializing as a table or view ![](/files/zIxxrwrqmYdv60LuGBVo) **Database (only for Snowflake)** Select the database you want to save to. Use the refresh icon if you believe that database or schema lists are not the latest ones. **Schema** Select a target schema from the list. **Name** Specify a table or view name. You can also toggle REWRITE to select from the existing views or tables. **Type and Save Options** The following materialization options are supported: * View (drop and create a new view on each run) * Table * Create (drop and create a new table on each run) * Append or update (append or update data on every run) * Incremental update (update table on each run using custom filters) ### Table materialization ![](/files/KlTzrQfInWpM6EYVXT0E) * **Drop and create** Drop the table if it exists and create a new one on each run. * **Append and update** Append or update rows in the existing table. If a table does not exist, create it. * **Incremental update** Append or update filtered rows in the existing table. If a table does not exist, create it. **Unique key (only for Append and Incremental Update options)** Optionally, if you can set a unique key, the records with the same unique keys will be updated. A unique key determines whether a record has new values and should be updated. Not specifying a unique key will result in append-only behavior, which means all filtered rows will be inserted into the preexisting target table without regard for whether the rows represent duplicates. ### Incremental Update The first time you execute a flow in Tomat, a new table is generated in your data warehouse by transforming the entire dataset from your source. For any subsequent runs, Tomat will only process and transform the specific rows you've chosen to filter, appending them to the already existing target table. Typically, you'll filter rows that have been added or updated since your last flow run. By doing so, you're minimizing the volume of data that needs to be transformed, which speeds up the runtime, enhances your warehouse's performance, and cuts down on computational expenses. **Condition to filter records (only for Incremental Update options)** Set a boolean condition to tell Tomat which rows to update or append on an incremental run. You'll often want to filter for "new" rows, as in rows created since the last time the flow was run. The best way to find the timestamp of the most recent run is by checking the most recent timestamp in your target table. Use `$target` to query to the target existing table. You can point to any column in the target table, adding a dot (.) to `$target -> $target.my_column` In the example below, only the following rows will be updated where `last_updated_time` are bigger than the maximum `last_updated_time` in the existing table. ```markup last_updated_time > max($target.last_updated_time) ``` # Chart Create a chart. ## Overview Chart node allows you to create informative charts based on your data so that you can visualize, gain insights, and design presentations.
## Settings There are three types of charts available. ![](/files/xes44MdB3xHoGpuqe1ac) ### Column and Bar Charts
In a bar chart, values are indicated by the length of bars, each corresponding with a measured group. Bar charts can be oriented vertically or horizontally; vertical bar charts are sometimes called column charts. Horizontal bar charts are a good option when you have many bars to plot or the labels on them require additional space to be legible. **Fields for column chart:** X-axis - represents categories (aka dimensions) that should be measured Y-axis - represents measure-numeric values, which could be aggregated. {% hint style="info" %} In a bar chart, the X-axis and Y-axis are swapped. {% endhint %} The **Aggregate Columns** option allows you to enable or disable aggregation. For example, the Chart node is built after [Group by](/data-transformation/transforms/group-by) node where aggregation has already been performed. Using the "+Add button," you can add another column for visualization on the chart. **Sort by** allows you to sort results. ### Line chart
Line charts show changes in value across continuous measurements, such as those made over time. Movement of the lineup or down helps bring positive and negative changes. It can also expose overall trends to help the reader make predictions or projections for future outcomes. **Fields**: X-axis - represents categories (aka dimensions) that should be measured Y-axis - represents measure-numeric values, which could be aggregated. Also, you can add more than one measure to the chart, shown in separate colored lines. ### Runtime When you run a job, charts are calculated on the whole dataset. You can find the resulting charts on the job page. ### Download charts ![](/files/eVutDbBFnUtMFPxNsA1q) You can download charts by clicking on the export icon in the footer. **Options**: * Copy as SVG to the clipboard * Download PNG file * Download SVG file # New Column Adds a new column with “text”, number, or function ## Overview The New Column Node allows you to add new columns to your dataset based on formulas, including existing column values and various functions.
## **Settings** You can create new columns, specify their names, and define the expressions or formulas to calculate the values for the new columns. #### **Set Value To** Use the "Set Value To" property to define the expression or formula for calculating the values in the new column. The expression editor provides autocomplete functionality, helping you easily access functions and column names. An expression can consist of: * text -> "my\_string" (always frame text with double quotes) * numbers -> 1.2 (use a period to separate the integral and fractional parts of decimal types.) * column names -> My\_Column (start typing column name and use auto-suggestions to add column reference) * functions -> Round(1.2234, 2) See the full list of operators in our tutorial [What are Formulas?](/data-transformation/formulas/what-are-formulas) #### **Column Name** The "Column Name" property allows you to specify the name for the new column. By default, the new column will be added to the dataset. If you want to overwrite an existing column instead, enable the "Overwrite" toggle and select the column you want to overwrite from the list. #### +Add Column To add more new columns, click the "Add Column" button. You can create multiple new columns with unique names and expressions. You can reference any columns added earlier in this node. ## **Preview** New columns will appear in the dataset, showing the calculated values based on the expressions or formulas you defined. # If...Then Adds a new column with a value based on the specified condition ## Overview The If...Then Node allows you to create or modify columns in your dataset based on conditional expressions. The node applies a value or expression to a new or existing column depending on whether the specified conditions are met.
## **Settings** #### **If** The "If" property allows you to define the condition for the If..Then Node. There are two options for creating conditions: 1. Predefined Operators. You can easily set up conditions based on column values and comparison operators using predefined operators. Follow these steps to create a condition: 1. Select the column you want to use as a basis for the condition. 2. Choose a comparison operator (e.g., equal, contains, etc) depending on the column's data type. 3. Enter a value to compare with the selected column's values. 2. Custom Formula. To use a custom formula, enable the "Custom Formula" toggle. With this option, you can define any condition that returns a boolean value (true or false). #### **Then set a value to** If the condition in the "If" property is met (returns true), the value or expression defined in the "Then Set Value To" property will be applied to the new or existing column. #### **+ Add Condition (Else If)** You can add an "Else If" block by clicking the "+Add Condition" button. This block will be evaluated if the previous condition(s) are not met (return false). If the "Else If" condition is met (returns true), its corresponding value or expression will be applied to the new or existing column. #### **Otherwise (if all conditions are failed)** If all conditions in the "If" and "Else If" blocks fail (return false), the value or expression specified in the "Otherwise" property will be applied to the new or existing column. #### **Column Name** The "Column Name" property allows you to specify the name for the new or existing column. By default, the new column will be added to the dataset. If you want to overwrite an existing column instead, enable the "Overwrite" toggle and select the column you want to overwrite from the list. ## **Preview** The new or modified column will appear in the dataset, showing the values or expressions applied based on the conditions you defined # Rolling Functions Calculates a window function operates on a group (“window”) of related rows ## **Overview** The rolling node allows you to perform calculations across a set of rows related to the current row. Unlike regular aggregate functions, which return a single value for a group of rows, window functions return a value for each row based on the rows within its "window."
{% hint style="info" %} Learn more in our tutorial: [Window Functions](/tutorials/window-functions) {% endhint %} ## Settings #### Set value to Select the column to calculate aggregation and choose one of the aggregation functions: Average, Count, Min, Max, Sum. #### Rows window This parameter defines the range of rows that should be included in the window frame relative to the current row and their sorting. **starts from - ends with** Specify the frame using one of the following options: * All preceding. Includes all rows from the start of the partition to the current row. * Preceding. Includes the previous **n** rows before the current row. * Current row. Includes the current row only. * Following. Includes the next **n** rows after the current row. * All following. Includes all rows from the current row to the end of the partition. **sorted by** This parameter is used to specify the order in which the window function will process the rows. Defining the order, especially when using ranking or numbering functions, is important as it directly impacts the output. #### Partitioned by This parameter divides the data into partitions to which the window function is applied. If you don't specify the "Partitioned by" clause, the function will treat the whole result set as a single partition. #### **Column name** Enter the name for a newly created column. # Column Type Changes a data type for selected column(s) ## Overview The Column Type Node allows you to easily modify the data types of specific columns in your dataset. Adding this node to your canvas allows you to select columns and convert them to integers, strings, dates, or other supported data types.
## Settings You can change the column type in two ways in the app: 1\. Using the toolbar, select the column you want to change the column type for and click on the 'Change column type' node 2\. From the table header, click on the column type icon and select the appropriate option ![](/files/bAFWe8akOw1GQNsxqE45) {% hint style="info" %} You can select multiple columns simultaneously and convert them to the same data type. {% endhint %} #### Change type for Select the columns you want to change the type for. All columns should have the same original formatting. #### Set new type #### String Simple text. You can convert any other type to a String. #### Integer A column type that stores whole number values without decimal points or fractions. Options: You can specify **a decimal separator in the original column** - period or comma, then decimal numbers will be truncated. #### Decimal A column that stores numeric values with an unfixed number of decimal places. You should specify **a decimal separator in the original column** - period or comma. {% hint style="info" %} All decimals are stored in databases as float type {% endhint %} #### Boolean A column that stores binary data representing *true* and *false*. You can convert to boolean columns containing the following values: * 'true', 't', 'yes', 'y', 'on', '1' return TRUE. * 'false', 'f', 'no', 'n', 'off', '0' return FALSE #### DateTime, Date, Time types DateTime stores both date and time information. Date stores only date part and Time - only time part. When converted all values in those columns will be displayed in ISO standard format: yyyy-MM-dd HH:mm:ss.ssa Example: ``` Datetime: 2023-08-17 10:30:45 Date: 2023-08-17 Time: 10:30:45 ``` You should specify the **format of the original column**. * Select one from the predefined list. See [supported](/data-transformation/formulas/supported-date-parts) abbreviations. Instead of "\*" could be the following symbols: "-", "\_", space * Switch the toggle **"custom"** to enter a custom formatting. #### Array A column that can hold multiple values or elements within a single cell allows for storing and manipulating structured data like lists, sets, or matrices, which can be useful for performing operations on groups of related information. Example: ``` ["2022-06-01", "2022-06-22", "DW", "inactive"] ``` #### Object A column that stores a collection of values, often of the same data type, grouped under a single variable. Example: {% code overflow="wrap" %} ``` {"End": "2022-06-22", "Start": "2022-06-01", "Status": "inactive", "Campaign name": "DW"} ``` {% endcode %} # Columns Edit Renames, deletes and moves columns ## Overview The Columns Node allows you to manage your dataset's columns by renaming, removing, and changing the order of columns using a simple drag-and-drop interface.
## **Settings** The Property Grid visually represents each column in your dataset as a plate displaying the column's name and data type. You can interact with these plates to perform the following actions: #### **Change Column Order** To change the order of the columns, click and drag the desired column plate to a new position in the Property Grid. The column will be moved to the new location when you release the mouse button. #### **Rename Column** To rename a column, click the pencil icon on the column plate or double-click the column name. This will make the column name editable. Type in the new column name, press Enter or click outside the input box to confirm the change. #### **Remove Column** Click the Trash icon on the column plate to remove a column from the dataset. # Sort Sorts a table by the specified column(s) ## Overview The Sort Node allows you to sort your dataset based on one or more columns, with a specified sorting direction for each column. ## **Settings** #### **Sort table by** Choose which columns you want to sort by. Press the “**+Then sort by**” link to add more columns to sort. **Sorting Direction** The available options are: 1. **Ascending**: Sort the column in ascending order, from the smallest to the largest value. 2. **Descending**: Sort the column in descending order, from the largest to the smallest value. # Filter Filters rows based on the specified condition ## Overview The Filter Node allows you to filter your dataset based on specific conditions or custom formulas. ## **Settings** #### **Find rows where** The "Find Rows Where" property allows you to define the conditions for filtering rows in your dataset. There are two options for creating conditions: **1. Predefined Operators** Using predefined operators, you can easily set up conditions based on column values and comparison operators. Follow these steps to create a condition: 1. Select the column you want to use as a basis for the condition. 2. Choose a comparison operator (e.g., equal, contains, more, etc.) depending on the column's data type. 3. Enter a value to compare with the selected column's values. To add more conditions, click the "Add Condition" button. You can combine multiple conditions using the following logical operators: * **And**: Both conditions must be true for a row to pass the filter. * **Or**: At least one of the conditions must be true for a row to pass the filter. You can also add a new group of conditions. Click the "Add Condition" button and select "Add group". **2. Custom Formula** To use a custom formula, enable the "Custom Formula" toggle. With this option, you can define any condition that returns a boolean value (true or false). #### **Action** The "Action" property lets you choose what to do with rows that pass the filter conditions. There are two available options: 1. **Keep Filtered**: Only keep rows that meet the filter conditions in the dataset. 2. **Remove Filtered**: Remove rows that meet the filter conditions from the dataset. # Remove Duplicates Removes duplicated rows ## Overview The Remove Duplicates Node allows you to identify and remove duplicate rows within your dataset. This node allows you to compare the entire rows or analyze specific columns to find duplicates. ## Settings There are two options for identifying duplicates: #### **Compare the whole row.** By selecting this option, the node will compare the entire row to identify duplicates. When all columns in a row have the same values as another row, it will be considered a duplicate. #### **Select columns to analyze duplicates.** This option allows you to specify which columns should be analyzed for duplicates. You can choose one or more columns to compare for duplicates. When you have chosen the columns, the node will only consider rows as duplicates if the selected columns have the same values in both rows. # Split Text Adds a new column with a value based on the specified condition ## Overview The Split Node allows you to split text from one or multiple columns in your dataset into separate columns or arrays based on a specified delimiter. ## **Settings** #### **Split text from** Select one or multiple columns you want to split in the "Split text from" property. The selected columns should contain text data that will be split using a specified delimiter. #### **By delimiter** Specify the delimiter to split the text using the "By delimiter" property. You can choose between a text delimiter or a regular expression (regex) delimiter: 1. Text delimiter: Enter a simple text string that will be used to split the column values. 1. Regex delimiter: Enter a regular expression (regex) pattern that will be used to split the column values. Find out how to use Regex [Using Regex](/tutorials/using-regex) and the list of supported tokens [Regex: List of Tokes](/data-transformation/formulas/regex-list-of-tokes) #### **Set into** In the "Set Into" property, choose one of the following options for the output format: #### **Array** If you select "Array," the Split Node will create a new column containing arrays with the split parts of the original text. #### **Columns** If you select "Columns," the Split Node will create separate columns for each split part of the original text. You need to specify the number of columns to be created. By default, two columns will be created. #### **Ignore case** When this option is enabled, the Split Node will not differentiate between uppercase and lowercase characters in the delimiter. # Extract Text Extracts the specified part of text into a new column(s) ## Overview The Extract Node allows you to extract specific text from one or multiple columns in your dataset and place the extracted text into separate columns or arrays. ## **Settings** #### **Extract From** In the "Extract From" property, select one or multiple columns that you want to extract text. The selected columns should contain text data that can be processed using the specified text or pattern. #### **Find** Specify the text or pattern to find and extract using the "Find" property. You can choose between a text string or a regular expression (regex) pattern: 1. Text string: Enter a simple text string that will be used to find and extract the text from the column values. 2. Regex pattern: Enter a regular expression pattern that will be used to find and extract the text from the column values. Find out how to use Regex [Using Regex](/tutorials/using-regex) and the list of supported tokens [Regex: List of Tokes](/data-transformation/formulas/regex-list-of-tokes) #### **Set into** In the "Set Into" property, choose one of the following options for the output format: #### **Array** If you select "Array," the Extract Node will create a new column containing arrays with the extracted text parts. #### **Columns** If you select "Columns," the Extract Node will create separate columns for each extracted text part. You need to specify the number of columns to be created. By default, two columns will be created. A maximum of 50 columns can be created. #### **Ignore case** Enable the "Ignore Case" toggle if you want the extraction process to be case-insensitive. When this option is enabled, the Extract Node will not differentiate between uppercase and lowercase characters when finding and extracting the text. # Find and Replace Text Finds and replaces the specified part of text ## Overview The Find and Replace Node allows you to search for specific text within one or multiple columns in your dataset and replace the found text with a specified value. In this guide, we will explain the properties and functionalities of the Find and Replace Nod ## **Settings** #### **Search into** In the "Search Into" property, select one or multiple columns to search for the specified text or pattern. The selected columns should contain text data that can be processed using the specified text or pattern. #### **Find** Specify the text or regex pattern to find using the "Find" property. You can choose between a text string or a regular expression (regex) pattern: 1. Text string: Enter a simple text string that will be used to find the text within the column values. 2. Regex pattern: Enter a regular expression pattern that will be used to find the text within the column values. Find out how to use Regex [Using Regex](/tutorials/using-regex) and the list of supported tokens [Regex: List of Tokes](/data-transformation/formulas/regex-list-of-tokes) #### **Replace with** Enter the replacement text in the "Replace With" property. This text will replace the found text or pattern within the selected columns. #### **Ignore case** Enable the "Ignore Case" toggle if you want the search process to be case-insensitive. When this option is enabled, the Find and Replace Node will not differentiate between uppercase and lowercase characters when finding the text. #### **Match all occurrences** The "Match All Occurrences" toggle is enabled by default. When this option is enabled, the Find and Replace Node will search for and replace all occurrences of the specified text or pattern within the selected columns. # Match Text Adds a new column with a value based on the specified condition ## Overview The Matches Node allows you to search for and count the number of matches of a specific text or pattern within a selected column in your dataset. ## **Settings** #### **Count matches into** In the "Count Matches Into" property, select a column you want to search for the specified text or pattern. The selected column should contain text data that can be processed using the specified text or pattern. #### **Find** Specify the text or regex pattern to find using the "Find" property. You can choose between a text string or a regular expression (regex) pattern: 1. Text string: Enter a simple text string that will be used to find the text within the column values. 2. Regex pattern: Enter a regular expression pattern that will be used to find the text within the column values. Find out how to use Regex [Using Regex](/tutorials/using-regex) and the list of supported tokens [Regex: List of Tokes](/data-transformation/formulas/regex-list-of-tokes) #### **Column name** Enter a name for the new column in the "Column Name" property. This column will store the count of matches found in the selected column. #### **Ignore case** Enable the "Ignore Case" toggle if you want the search process to be case-insensitive. When this option is enabled, the Matches Node will not differentiate between uppercase and lowercase characters when finding the text. # Join Joins 2 tables using the specified columns as keys ## Overview The Join Node allows you to combine two datasets based on one or more shared keys. This is particularly useful for scenarios where you want to merge data from different sources to create a comprehensive view. It's similar to VLookup in Excel but more comprehensive. Read more about in our tutorial [Join Types](/tutorials/join-types)

Select Join node on toolbar

## Settings **Left and Right tables** Select two nodes to join. Left and Right join types will work depending on nodes selection in this section. #### Types of Joins Tomat supports the following types of joins: * **Left Join**: Includes all records from the left table and the matching records from the right table. * **Right Join**: Includes all records from the right table and the matching records from the left table. * **Inner Join**: Includes only the records with matching keys in both tables. * **Full Join**: Includes all records when a match occurs in either the left or right table. * **Cross Join**: Combines all records from both tables. {% hint style="info" %} Be cautious with "Cross Joins" as they can result in many records. {% endhint %} #### Keys Key columns are the basis for matching records between the two datasets you're joining. A key is a specific field (or fields) in each table used to align the data. These columns contain unique identifiers or attributes in both datasets, allowing to match rows from one table to another. Select keys for left and right tables from dropdowns. You can add more than one key pair with the **"+ Add"** link. {% hint style="info" %} When you have multiple keys, the join operation will consider all keys for matching records. {% endhint %} #### Output columns Uncheck the columns you do not want to see in the result table. You can use tabs "Left table" and "Right table" to filter columns by table quickly. # Union Stacks rows of two or more tables ## Overview A Union node allows you to combine rows from different tables into a single result set or “stack” tables one on another. Tomat Union works similarly to the SQL UNION operator but with more flexibility. It can combine tables with different column sets by matching columns based on their names and data types. The column order in the original tables does not matter. Learn more in out tutorial [Union Introduction](/tutorials/union-introduction)
## Settings {% hint style="info" %} You can add a Union node from the toolbar or select multiple tables on canvas (Shift + Left mouse button -> drag selection) and add it from the Action panel. In the second case, you can quickly union many tables in one step. {% endhint %} #### **Tables to stack** Select the tables you want to stack. Columns will be mapped automatically based on their Names and Types, and column order is not considered. If a column exists in one table but not in another, the resulting table will include this column, but it will be empty for the table lacking it. You can add multiple tables with the **"+Add a table"** link. #### Result columns Uncheck the columns you do not want to see in the result table. Use colored marks to understand in which input tables columns exist. # Group By Groups rows and calculate aggregation functions ## Overview The Group By Node allows you to easily group your data by one or more columns and perform various aggregate functions such as sum, average, or count. This functionality is crucial for generating summarized views of your data and is often a vital step before further analysis or visualization.
## Settings #### Group rows by Select a column to group by. To choose multiple columns, click the "+ Then group by" link and select another one. Additionally, you can tailor the sorting order for each column individually. #### Set values to Select columns to perform aggregation. You can select multiple columns with the "+ Add value" link. {% hint style="info" %} Read more about the available aggregation functions [here](/data-transformation/formulas/aggregate-functions). {% endhint %} There are two options for creating aggregations: 1. Predefined aggregation functions. Follow these steps: 1. Select the column you want to use as a basis for the aggregation. 2. Choose an aggregation function from the list. 2. Custom Formula. Enable the "Custom Formula" toggle to use a custom formula. With this option, you can define any custom expression that should return an aggregation function.
# Pivot Creates new columns from values in the specified columns and computes aggregation functions as values for the new columns ## Overview The Pivot Node allows you to convert rows into columns and computes aggregate functions as values for the new columns. This transformation is particularly useful for creating cross-tabular views, making comparing and contrasting different data dimensions easier. The reverse operation is [Unpivot](/data-transformation/transforms/unpivot).
## Settings ![](/files/xpZ2xz05coIwajssjVfX) #### Create columns from Select a column from which values you want to create new columns. The Pivot node will create a distinct column for each unique value in the selected column. #### Group rows by Select the columns you want to group by. Column order determines group hierarchy. #### Set values to Select columns to perform aggregation operations. You can select multiple columns for aggregation. There are two options for creating aggregations: 1. Predefined aggregation functions. Follow these steps: 1. Select the column you want to use as a basis for the aggregation. 2. Choose an aggregation function from the list. 2. Custom Formula. Enable the "Custom Formula" toggle to use a custom formula. With this option, you can define any custom expression that should return an aggregation function. #### Place values as Applied only if more than one aggregated column was selected. Select "Columns" to create a new column for each aggregated column, and "Rows" to create new rows for the aggregations instead of columns. # Unpivot Reshapes the data by merging one or more columns into key and value columns ## Overview The Unpivot node allows you to convert columns in your data table into row values. It involves converting wide-format data with multiple columns for different categories or attributes into long-format data, where each category or attribute is represented in a single row.
## Settings #### **Convert to rows** Select the columns that you want to unpivot into rows. All columns' names will be stored in one column and their values in another. #### **Category column name** Enter a name for the new column to store the names of the unpivoted columns. #### **Value column name** Enter a name for the new column to store the values from the unpivoted columns. #### **Keep all other columns** Switch this toggle "on" to keep columns that were not unpivoted. ## **Example** Consider the following wide-format dataset:
CountryYearProduct_A_SalesProduct_B_SalesProduct_C_Sales
USA2021100020001500
UK202180017001200
France202190019001300
The result of Unpivot: | Country | Year | Product | Sales | | ------- | ---- | ---------- | ----- | | USA | 2021 | Product\_A | 1000 | | USA | 2021 | Product\_B | 2000 | | USA | 2021 | Product\_C | 1500 | | UK | 2021 | Product\_A | 800 | | UK | 2021 | Product\_B | 1700 | | UK | 2021 | Product\_C | 1200 | | France | 2021 | Product\_A | 900 | | France | 2021 | Product\_B | 1900 | | France | 2021 | Product\_C | 1300 | # To JSON Creates Objects or Arrays in JSON format from the specified columns ## Overview The To JSON Node provides a powerful way to combine multiple columns into a single column with structured data, such as arrays or objects. This is especially useful when dealing with hierarchical or relational data and wanting to keep related values bundled together.
## Settings ![](/files/fO2QCuKwd1HsYTACY36B) #### Combine values from Select one or multiple columns that you want to combine into JSON. #### Into Choose a type of JSON to combine all columns into: an array or object. Object. When an object type is selected, the column name is used as a key name, and column values as values. ``` { "Gender": "M", "Country": "USA", "BirthYear": 1980 } ``` Array. If you select array type information about column names will be lost. ``` ["USA", "M", 1980] ``` #### Column name Enter a name for the new column or select the column you want to overwrite. If you want to overwrite values in an existing column, enable toggle **Overwrite** and select the required column. # From JSON Flatten JSON to columns and rows ## Overview The From JSON Node allows you to expand structured data fields, like arrays or objects, back into individual columns and rows. Adding this node lets you deconstruct complex column structures to enable easier analysis and reporting. The opposite transform is [Nest](/data-transformation/transforms/to-json).
## Settings #### Extract data from Select columns containing JSON you want to expand. Columns should have an Array or Object type. If more than one column is selected, all the columns should have the same structure and type. #### Options Select an option to automatically extract values from JSON or set manual paths to specific values. **Unnest top level** This option automatically recognizes the column type and extracts Objects into columns and Arrays into rows. **Unnest only objects and values** All arrays will be extracted as it is, with no parsing. **Custom flatten with JsonPaths** Specify the paths manually using JSON Path language. Set the cursor inside the field to get field suggestions. Select the field or press "Tab" to add it to the path. ![](/files/J5WzcP4Stj2VorrViFfe) #### Compare the Options Consider the selected column contains the following JSON: {% code overflow="wrap" %} ```json { "Name": { "FirstName": "Mike", "SecondName": "Blank", }, "Nationality": ["USA", "Poland"], "BirthYear": 1980 } ``` {% endcode %} The result of "Unnest top level": | Name | Country | BirthYear | | ------------------------------------------------- | ------------------ | --------- | | { "First Name": "Mike", "Second Name": "Blank", } | \["USA", "Poland"] | 1980 | The result of "Unnest only objects and values": | Name.FirstName | Name.SecondName | Country | BirthYear | | -------------- | --------------- | ------------------ | --------- | | Mike | Blank | \["USA", "Poland"] | 1980 | # API Call Calls an external API and return a new column with answers ## Overview The API Call node facilitates interaction with external APIs, allowing you to extend your data transformation and automation capabilities. You can invoke an external API for each dataset row and capture the response in a new column. This is particularly useful when you need to enrich or validate your data using third-party services or proprietary APIs.
## Settings #### Authorization Select an authentication method for your API call. This method will be included in the HTTP headers to authenticate your requests with the external API. * None. No authentication. * Bearer Token. When selected, you must provide a bearer token. * Basic Auth. When selected, you should provide a pair user name and password, or only the password. #### Method The "Method" field allows you to specify the HTTP method for the API call. Common methods include GET, POST, PUT, DELETE, etc. The method you choose should match the endpoint's requirements you are interacting with. #### URL In the "URL" field, you enter the endpoint of the external API you wish to interact with. This is the web address to which the API call will be sent. Use a sign `@` to mention existing columns.
#### Headers In the "Headers" field, you can specify additional HTTP headers to be included in the API request. Headers may be used to provide extra information about the request or to set certain conditions for processing the request. #### Body type and Content type * **Body type**: This field allows you to select the format of the request body, such as Raw, Form-Data, x-www-form-urlencoded, etc. The selection here depends on the requirements of the API you're interacting with. * **Content type**: This field specifies the media type of the request body, such as application/json, application/xml, text/plain, etc. It tells the API how to interpret the contents of the request body. {% hint style="info" %} Now, we support only raw: application/json type and plain text. {% endhint %} #### Request content The "Request Content" field is where you input the data or parameters you want to send in the body of the API request. The format of this content should match the "Body Type" and "Content Type" settings. For the current application version, it should be empty or in JSON format. Use a sign `@` to mention existing columns. #### Button "One row" You can call API row by row here. We recommend always calling external API for one row fistly to be sure you get an expected result. #### Button "All rows" If you are good with one-by-one calling, you can call API for all rows simultaneously. It will not recall API for those rows already containing the result. #### Button "Remove answers" If you get some errors in the answer or want to change the request - use this button to remove results for all rows. # Use AI Calls AI for each row to extract, enrich or cleanup. ## Overview The Use AI node helps you run an AI model on your table, row by row. You write a prompt, pick a model, and the node returns the results for every row in new columns. ## How It Works ### **1. Write a Prompt** Describe what you want the AI to do for each row in the prompt box. Use `@ColumnName` to pull in values from your table. **Example**: {% code overflow="wrap" %} ``` Visit @CompanyDomain and find their ICP titles. Return them as a comma-separated string. ``` {% endcode %} ### **2. Choose a Model** Select the AI model you want to use from the drop-down menu. Not all models support web search ### **3. Web Search (Optional)** Turn on web search if your prompt needs fresh information from the internet. If the chosen model does not support web search, this option will be disabled. Set the search depth to Low, Medium, or High: * **`high`** Most comprehensive context, highest cost, slower response. * **`medium`** (default) Balanced context, cost, and latency. * **`low`** Least context, lowest cost, fastest response, but potentially lower answer quality. ### **4. Set Output Columns** Add as many output columns as you need. For each column, set its name, type (String, Integer, Boolean, etc), and optional description to help AI. You can use “Generate from prompt” to let the AI suggest columns based on your prompt. If you don’t add any columns, the node will automatically create them based on the AI’s first response. Switch to JSON schema view if you want to see or edit all columns as JSON. ### **5. Execute the Prompt** You can run the node on all rows, just one row, or selected rows. An alternative way to get the result for a specific row is to find any output column in the table preview and click the "Generate row" link. The node works row by row, passing each row’s values to the prompt and writing back the results. ### **6. Edit or Regenerate Rows** After the run, you can manually edit, add, or delete values in the output columns. You can also re-run the node for any specific rows directly from the table, without affecting other rows. ## Tips for Best Results **Test your prompt** Start by running the node on one or a few rows to see if you get the desired output. Adjust your prompt or output columns before running it on the whole table. We recommend always using this option first. When you are ready, press the "All rows" button to return values for all remaining rows. It will not refill rows already containing the result. **One row at a time** The node works with only one row at a time. It cannot calculate aggregations, compare, or group rows. **Changing output columns** If you want to change the output columns after a run, reset the results. **Web search and credits** Using a web search will use more credits. Only turn it on if your prompt actually needs it. **Model limits** Each model has its limit for how much text it can process. Try a shorter prompt or pick a different model if you hit an error. **Show only affected columns.** Use the "Show only affected columns" toggle below the table to show only the columns used in the prompt or set as output columns and hide all other columns. # AI Table Creates a new table based on GPT prompt ## Overview By adding AI Table Node, you can execute complex transformations over single or multiple tables with just a natural language prompt. It is a revolutionary productivity tool designed to simplify the creation of complex data transformations - it allows you to condense multiple transformation steps into a single node.
## Settings #### Input nodes Select the nodes you want to work with. The "+Add node" link allows you to add multiple nodes and select a node on the canvas.
Assign them custom names to make it simpler to refer to when constructing the prompt. ![](/files/WwzjECnVKDrgBjpN3IU2) #### Prompt Write your prompt in natural language to describe transformations you want to do over input datasets. You could mention the existing columns using `@` symbol. {% hint style="warning" %} AI doesn't know the data in the columns. It only knows the schemas of the selected nodes (column names and types) and one row of data. That means you cannot ask it to calculate results based on knowledge about values in the table. But you **can** ask anything if it can be done using standard functions. {% endhint %} Click the submit button to send your prompt to AI. AI returns a completely new table based on your request.
If AI returns incorrect results, try rewriting the prompt using more details. For example, you could specify the names of columns as context. You can also try different approaches to making a prompt to achieve the desired result. ### **What to ask** * Any manipulation with input datasets such as join, group, summarize, filter, sort * Find the top 5 most active users by the number of events * Filter users with empty names * Return user numbers by country codes, filter top 10 * Group users by industry * As AI knows nothing about the data in the tables, it cannot answer the questions like: * Find negative reviews * Filter users with inactive status (for example, if the status can be “on hold”, “inactive”, “or removed” - all inactive - AI doesn't know that you need specifically mention all these statuses) * Find all users from NY (again, AI doesn't know the data in the city column and doesn't understand how to create a filter: “NY,” “New York,” “Apple City”) # Formulas # What are Formulas? Tomat's formula language has a lot to offer, with hundreds of functions quite similar to your favorite ones in Excel and SQL. Formulas can be as simple as a single number or as complex as a function call or mathematical formula. ### Operators * Regular math operators: `+`, `-`, `*`,`/`,`%` * Comparison operators (evaluate to booleans): `>`, `>=`, `<`, `<=`, `==`, `!=`, `<>`. These are used to compare two values and produce a boolean result (true or false). * Logical operators: `!` , `&&` , `||` (also support `not`, `and`, `or` variation). These operators are used to manipulate boolean values. * String concatenation operators: `|` . This is used to combine two or more strings into one. ### Expressions * Literal: represents a fixed value, like a number, "string", boolean (true, false), or a reference. * Unary: composed of a single operand and an operator. The operator can precede the operand `-expression`, `!expression` * Binary: consists of two operands and an operator `expression`` `*`operator`*` ``expression`. * Grouping: expressions can be grouped using parentheses **`(`**`expression`**`)`** to control the precedence of evaluation. ### References References in expressions can be made to: * Column by name: this allows for operations on specific columns in a data table. * REGEX Path to values in the specified columns: this allows for complex data manipulation using regular expressions. * Constants and enums: these can be used as parameters in functions. # Math Functions # Abs Returns the absolute value of a numeric expression {% code overflow="wrap" lineNumbers="true" fullWidth="false" %} ```mathematica Abs(24) → 24 Abs(-17) → 17 ``` {% endcode %} #### Inputs `Abs(number)` `number`- A number expression # Ceiling The Ceiling function rounds a number up to the nearest integer multiple of specified significance. {% code overflow="wrap" lineNumbers="true" fullWidth="false" %} ```nb Ceiling(3.14, 0.1) → 3.2 Ceiling(7, 3) → 9 ``` {% endcode %} #### Inputs `Ceiling(value, [factor])` `value` - The value to round up to the nearest integer multiple of `factor`. `[factor]` - \[optional: `1` by default] - The number to whose multiples `value` will be rounded. `factor` may not be equal to `0` # Exp Returns Euler's number e (\~2.718) raised to a power ```mathematica Exp(1) → 1 Exp(2) → 7.38 ``` #### Inputs `Exp(number)` `number`- A number expression #### Output Outputs Euler's number for `number` e (\~2.718) raised to a power. # Floor Rounds a number down to the nearest integer multiple of specified significance. ```mathematica Floor(30.527, 0.01) → 30.52 Floor(1.15, 1) → 1 ``` #### **Inputs** `Floor(value, [factor])` * `value` - The value to round down to the nearest integer multiple of `factor`. * `[factor]` - **\[optional**: `1` by default\*\*]\*\* - The number to whose multiples `value` will be rounded. `factor` may not be equal to `0` # IsEven Checks if a value is even ```mathematica IsEven(4) → true IsEven(3) → false ``` #### **Inputs** `IsEven(number)` * `number`- A number expression Return `true` if a `number` is even, and `false` otherwise # IsOdd Checks if a value is odd ```mathematica IsOdd(4) → false IsOdd(3) → true ``` #### **Inputs** `IsOdd(number)` * `number`- A number expression Return `true` if a `number` is odd, and `false` otherwise # Ln Returns the natural logarithm of a number (base e) ```mathematica Ln(17) → 2.83 Ln(2.72) → 1.00 ``` #### **Inputs** `Ln(number)` * `number`- A number expression # Log Returns the logarithm of a numeric for the specified base ```mathematica Log(25, 5) → 2 Log(20, 3.1) → 2.64 ``` #### **Inputs** `Log(number, base)` * `number`- A number expression * `base` - Logarithm base to use # Log10 Returns the logarithm of a numeric for the base 10 ```mathematica Log10(100) → 2 Log10(2.5) → 0.39 ``` #### **Inputs** `Log10(number)` * `number`- A number expression # Mod Returns the modulo value, which is the remainder of dividing the first argument by the second argument ```mathematica Mod(20, 6) → 2 Mod(100, 10) → 0 ``` #### **Inputs** `Mod(dividend, divisor)` * `dividend` - A number expression * `divisor` - A number expression # Pi Returns the constant that represents pi (\~3.14) ```mathematica Pi() → 3.14 ``` #### **Inputs** `Pi()` # Power Calculates a number raised to a power ```mathematica Power(4, 2) → 16 Power(2, -1) → 0.5 ``` #### **Inputs** `Power(base, exponent)` * `base` - The number to raise to the `exponent` power. If `base` is negative, `exponent` must be an integer. * `exponent` - The exponent to raise `base` to. # Quotient Quotient of the division of 'dividend' by 'divisor’ ```mathematica Quotient(50, 2) → 25 ``` #### **Inputs** `Quotient(dividend, divisor)` * `dividend` - A number expression * `divisor` - A number expression # Round Rounds a number ```mathematica Round(99, 6) → 100 Round(99.65, 1) → 99.7 ``` #### **Inputs** `Round(number, [places])` * `number` - A number to round * `[places]` - **\[optional**: `0` by default\*\*] -\*\* The number of decimal places to round to # RoundDown Rounds a number down ```mathematica RoundDown(98.57) → 98 RoundDown(98.57, 1) → 98.5 ``` #### **Inputs** `RoundDown(number, [places])` * `number` - A number to round down * `[places]` - **\[optional**: `0` by default\*\*] -\*\*The number of decimal places to round to. # RoundUp Rounds a number up ```mathematica RoundUp(28.2) → 29 RoundUp(28.22, 1) → 28.3 ``` #### **Inputs** `RoundUp(number, [places])` * `number` - A number to round up * `[places]` - **\[optional**: `0` by default\*\*] -\*\* The number of decimal places to round to. # Sign Returns the sign of a number. ```mathematica Sign(5) → 1 Sign(-5) → -1 ``` #### **Inputs** `Sign(number)` * `number` - A number expression #### **Outputs** Outputs -1 if number is negative, 0 if number is zero, or 1 if number is positive. # Sqrt Returns the square root of a non-negative numeric ```mathematica Sqrt(9) → 3 Sqrt(30) → 5.47 ``` #### Inputs `Sqrt(number)` * `number` - A number expression # Truncate Truncate a number ```mathematica Truncate(1.59, 1) → 1.5 Truncate(6.2) → 6 ``` #### Inputs `Truncate(number, [places])` * `number` - A number expression * `[places]` - **\[optional**: `0` by default\*\*] -\*\* The number of decimal places to truncate at. # Trigonometric Functions # Acos Computes the inverse cosine (arc cosine) of its input; a result is a number in the interval \[0, pi] ```mathematica Acos(1) → 0 Acos(-0.5) → 2.09 ``` #### Inputs `Acos(number)` * `number` - A number expression # Asin Computes the inverse sine (arc sine) of its argument; a result is a number in the interval `[-pi/2, pi/2]` ```mathematica Asin(0.5) → 0.52 Asin(1) → 1.57 ``` #### Inputs `Asin(number)` * `number` - A number expression # Atan Computes the inverse tangent (arc tangent) of its argument; the result is a number in the interval `[-pi, pi]` ```mathematica Atan(1) → 0.78 Atan(0.3) → 0.29 ``` #### Inputs `Atan(number)` * `number` - A number expression # Atan2 Computes the inverse tangent (arc tangent) of the ratio of its two arguments. For example, if x > 0, then the expression `ATAN2(y, x)` is equivalent to `ATAN(y/x)` ```mathematica Atan2(1, 2) → 1.10 Atan2(100, 50) → 0.46 ``` #### Inputs `Atan2(x,y)` * `x` - A number expression * `y` - A number expression # Cos Computes the cosine of its argument; the argument should be expressed in radians ```mathematica Cos(3.14) → -0.99 Cos(0) → 1 ``` #### Inputs `Cos(number)` * `number` - A number expression, should be expressed in radians # Cot Computes the cotangent of its argument; the argument should be expressed in radians ```mathematica Cot(3.14/4) → 1.00 Cot(1.02) → 0.61 ``` #### Inputs `Cot(number)` * `number` - A number expression, should be expressed in radians # Degrees Converts radians to degrees ```mathematica Degrees(3.14/2) → 89.95 Degrees(3.14/4) → 44.97 ``` #### Inputs `Degrees(number)` * `number` - A number expression # Radians Converts degrees to radians ```mathematica Radians(180) → 3.14 Radians(90) → 1.57 ``` #### Inputs `Radians(number)` * `number` - A number expression # Sin Computes the sine of its argument; the argument should be expressed in radians ```mathematica Sin(3.14/2) → 0.99 Sin(2) → 0.90 ``` #### Inputs `Sin(number)` * `number` - A number expression, should be expressed in radians # Tan Computes the tangent of its argument; the argument should be expressed in radians ```mathematica Tan(-3.14/4) → -0.99 Tan(-3.14/2) → -1,255.76 ``` #### Inputs `Tan(number)` * `number` - A number expression, should be expressed in radians # String Functions # Compare Compares two string values ```mathematica Compare("text1", "Text1", true) → false Compare("Mac", "Mac") → true ``` #### Inputs `Compare(string1, string2, [isCaseSensitive])` * `string1` - The first string to compare with * `string2` - The second string to compare with * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if the search should be case **insensitive** # Concat Merge multiple string values ```mathematica Concat("Meeting on ", Today()) → "Meeting on 2023-01-20" Concat("Tomat ", "app") → "Tomat app" ``` #### Inputs `Concat(string1, [string2...])` * `string...` - Text values, one or more separated by a comma # Contains Returns true if a string contains a search ```mathematica Contains("WINDOWS", "OWS") → true Contains("12345", "6") → false Contains(column, "WBC") → true ``` #### Inputs `Contains(string, search)` * `string` - A text value to search in * `search` - A text value to search for # In Returns `true` if a string is one of the following values. The `In` function is a shorthand for multiple `or` conditions. ```mathematica In("Germany", "USA", "Austria", "Germany") → true In("Germany", "USA", "Austria", "Mexico") → false In(column, "USA", "Austria", "Germany") → true ``` #### Inputs `In(string, [value...])` * `string` - A text value to search for * `value` - List of values to compare with the string # CountMatches Finds and counts all string matches ```mathematica CountMatches("WINDOWS", "W") → 2 CountMatches("1 2 3 4 5 6", 7) → 0 ``` #### Inputs `CountMatches(string, search, [isCaseSensitive])` * `string` - A text value to search in * `search` - A text value to search for * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** # CountMatchesRegexp Regexp Finds and counts all regular expression matches ```mathematica CountMatchesRegexp("MAC", "[A-Z]") → 3 CountMatchesRegexp("1234", "[5-9]") → 0 ``` #### Inputs `CountMatchesRegexp(string, searchRegexp, [isCaseSensitive])` * `string` - A text value to search in * `searchRegexp` - A regular expression to search for * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** # EndsWith Checks if a value ends with a search string ```mathematica EndsWith("Australia", "ia") → true EndsWith("Financial position", "e") → false ``` #### Inputs `EndsWith(string, search, [isCaseSensitive])` * `string` - A text value to search in * `search` - A text value to search for * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** # EndsWithRegexp Checks if a value ends with a search regular expression ```mathematica EndsWithRegexp("Financial performance", "[abc]") → false EndsWithRegexp("Dollars (millions)", "[)/]") → true ``` #### Inputs `EndsWithRegexp(string, searchRegexp, [isCaseSensitive])` * `string` - A text value to search in * `searchRegexp` - A regular expression to search for * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** # Extract Extracts the specified regular expression patterns from a string to an array ```mathematica Extract("code: 650-33", "[0-9]{3}", "^", "$") → "650" Extract("text text", "[o-y]", "[e]", "[xt]") → "x" Extract("123 056", "[2-5]", "[1]", "[6]") → 2 ``` #### Inputs `Extract(string, searchRegexp, startAfterRegexp, endBeforeRegexp, [index])` * `string` - Any text value to extract from * `searchRegexp` - Regular expression to find to * `startAfterRegexp` - Regular expression to start the search after * `endBeforeRegexp` - Regular expression to end the search before * `[index]` - **\[optional**: `O` by default\*\*]\*\* # FindMatchOfString Finds for “index”-th match of string pattern in a string value ```mathematica FindMatchOfString("text text", "ext") → "ext" FindMatchOfString("Hello WOrld", "l") → "l" ``` #### Inputs `FindMatchOfString(string, search, [isCaseSensitive], [index])` * `string` - Any text value to search in * `search` - Text to search for * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** * `[index]` - **\[optional**: `O` by default\*\*]\*\* # FindMatchOfRegexp Finds for “index”-th match of regular expression pattern in a string value ```mathematica FindMatchOfRegexp("text text", "[a-m]") → "e" FindMatchOfRegexp("Hello world", "[l]") → "l" ``` #### Inputs `FindMatchOfRegexp(string, searchRegexp, [isCaseSensitive], [index])` * `string` - Any text value to search in * `searchRegexp` - A regular expression to find to * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** * `[index]` - **\[optional**: `0` by default\*\*]\*\* # FindMatchesOfString Finds all matches of string pattern in a string value ```mathematica FindMatchesOfString("Hello World","l") → ["l","l","l"] FindMatchesOfString("text text","ext") → ["ext","ext"] ``` #### Inputs `FindMatchesOfString(string, search, [isCaseSensitive])` * `string` - Any text value to search in * `search` - Text to search for * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** # FindMatchesOfRegexp Finds all matches of regular expression pattern in a string value ```mathematica FindMatchesOfRegexp("Hello world", "[wo]") → ["o","w","o"] FindMatchesOfRegexp("text text", "[t]") → ["t","t","t","t"] ``` #### Inputs `FindMatchesOfRegexp(string, searchRegexp, [isCaseSensitive])` * `string` - A text value to search in * `searchRegexp` - A regular expression to find to * `[isCaseSensitive]` - **\[optional**: `true` by default\*\*]\*\* - Set `false` if search should be case **insensitive** # Left Returns a leftmost substring of a string value ```mathematica Left("Central region", 7) → "Central" Left("New York City", 8) → "New York" ``` #### Inputs `Left(string, length)` * `string` - A text value * `length` - Length of the resulting text --- [Next Page](/llms-full.txt/1)