Hadoop for Beginners (7): HDFS Data Read and Write Flow
Last time we could already operate HDFS from code, but what actually happens inside the Hadoop cluster? This post gives a quick overview of how HDFS reads and writes data.
TagAll posts
#Ops
173 posts
More Posts
Page 5 of 18Hadoop for Beginners (6): Using the Hadoop API to Drive HDFS from Code
Last post covered operating HDFS with shell commands, but in practice we can't keep doing everything by hand — we need automation through code. This post gets you familiar with controlling files in HDFS from Java.
Hadoop for Beginners (5): HDFS, the Distributed File System
HDFS stands for Hadoop Distributed File System. It is a highly fault-tolerant system designed for deployment on cheap commodity machines. It delivers high-throughput data access, which makes it a great fit for applications over very large data sets.
Hadoop for Beginners (4): A Fully Distributed Hadoop Cluster
Last time we tried installing a pseudo-distributed setup, but what you actually use in production is a distributed cluster — that's the real point. This post builds a minimal Hadoop cluster on three nodes to experience a fully distributed environment.
Hadoop for Beginners (3): Single-Node Local and Pseudo-Distributed Mode
Since this is a beginner course and many people's machines can't handle a real cluster, let's start with Hadoop's single-node mode so you can run simple operations with Hadoop MapReduce and HDFS. This section is basically a Hello World for Hadoop — pulling the mystery off it.
Hadoop for Beginners (2): Installing Hadoop
Before we can start our Hadoop journey we need to know how to install it. Later we'll experiment and learn across multiple Hadoop nodes; this post walks through the installation, which is quite simple.
Connecting DBeaver to a DaMeng Database
Newer DBeaver versions no longer support ODBC connections, probably because the JDBC-ODBC bridge was removed from Java 8. So the old tutorials telling you to use ODBC no longer work. Here's the correct way to connect DBeaver to DaMeng.
Making MariaDB/MySQL Case-Insensitive (Ubuntu on Linux)
MariaDB/MySQL is case-sensitive on Linux, so any mismatch in capitalization between your table or column names and your SQL throws errors. That's why we usually turn case sensitivity off first.
Hadoop for Beginners (1): What Hadoop Is and What It's Made Of
The previous big chapter covered the concept of a big data warehouse and how to think about building one. Now we start turning those ideas into reality. The foundation carrying all of it is the collection of big data components in the Hadoop ecosystem, which gradually becomes our data warehouse and platform.
Data Warehouse Concepts (4): Full, Incremental, Zipper, Transaction and Snapshot Tables
Last post covered fact tables, dimension tables, the star schema and the snowflake schema. Beyond those there's more industry jargon to learn. This post explains the big data slang: full tables, incremental tables, zipper tables, transaction tables, and snapshot tables. Some of it may not click yet — it will once you work with Hive — so take this as foundational knowledge.
