What is Farcaster hub sync
Hub sync is the process by which Farcaster hubs replicate data to stay consistent with the rest of the network. It is not merely copying files; it is a continuous state reconciliation that ensures every hub holds the same view of user data, such as casts, reactions, and profiles.
The network relies on a delta-based sync model. Instead of re-downloading the entire dataset from scratch, hubs exchange only the "deltas"—the incremental changes that have occurred since the last synchronization. This approach minimizes bandwidth usage and allows the network to scale as the volume of messages grows.
When a hub starts or reconnects, it identifies the gaps in its local database and requests the specific missing deltas from its peers. This mechanism ensures that new data propagates efficiently while preserving the integrity of the decentralized social graph.
Set up your hub node
Before you can sync data, you need a reliable machine running the Farcaster Hub software. Your node acts as the foundation for your participation in the network, storing and validating messages. A stable connection and sufficient resources are non-negotiable for keeping your data in sync with the rest of the network.
Choose the right hardware
Farcaster Hub is resource-intensive. The database grows by approximately 100-200 MB per day, depending on network activity. You need a machine with at least 16 GB of RAM and a fast NVMe SSD with at least 1 TB of free space. Cloud providers like AWS, DigitalOcean, or Linode work well, but ensure your instance type supports high I/O operations. A home server can work if you have the bandwidth and storage, but cloud instances offer more predictable uptime.
Install the hub software
The Farcaster Hub is written in TypeScript and runs on Node.js. You will need Node.js version 18 or higher installed on your system. You can install the hub globally using npm or yarn. Once installed, you can verify the installation by running the hub command to check the version. This step ensures your environment is ready to execute the sync process.
Configure and initialize
Before starting the sync, you must configure the hub. Create a configuration file or environment variables to set your node name, peer list, and data directory. The hub needs to know where to store the database and which peers to connect to initially. You can find a list of recommended peers in the official Farcaster documentation. Run the initialization command to create the necessary database files. This prepares the local storage for the incoming data stream.
Start the sync process
Launch the hub with the start command. The node will begin connecting to peers and downloading the latest state. The initial sync can take several hours or even days, depending on the size of the database. Monitor the logs to ensure the node is connecting successfully and processing blocks. Once the sync is complete, your node will be fully operational and ready to serve data.
Connect to the mainnet peers
Once your hub software is running, the next step is to tell it where to find the rest of the network. Farcaster relies on a distributed network of hubs that sync data with each other. To start receiving messages, you must configure your hub to connect to a set of known, reputable mainnet peers.
These peers act as the initial entry points to the network. By connecting to them, your hub begins the process of downloading the latest frames and messages, eventually building out its own local copy of the Farcaster graph. Without these connections, your hub remains isolated and cannot serve or sync data.
1. Locate the official mainnet peer list
The Farcaster team maintains a curated list of mainnet peers. This list is the most reliable source for finding stable, high-quality nodes. You can find the current list in the official Farcaster documentation or the farcaster GitHub repository.
Look for a file or section labeled mainnet-peers.json or similar. This file contains a list of peer addresses in the format enode://... or ip:port. Keep this list handy for the next step.
2. Edit your hub configuration file
Open the configuration file for your hub software. This is typically a YAML or JSON file located in your hub's installation directory. Look for a section related to peers, bootstrap, or sync.
If the section does not exist, you may need to add it. The structure will vary slightly depending on the hub implementation (e.g., Hubble, Farcaster Hub, etc.). Ensure you are editing the correct file for your specific software version.
3. Add the peer addresses
Copy the peer addresses from the official list and paste them into your configuration file. Each address should be on its own line or in its own array element, depending on the file format.
For example, in a YAML file, it might look like this:
peers:
- "enode://..."
- "enode://..."
In a JSON file, it might look like this:
{
"peers": [
"enode://...",
"enode://..."
]
}
4. Restart the hub
After saving your changes, restart the hub service. This forces the hub to reload the configuration and attempt to connect to the new peers.
Monitor the logs for connection messages. You should see entries indicating that the hub is connecting to the peers and starting the sync process. If you see errors, double-check the peer addresses and your network settings.
5. Verify the sync
Once the hub has been running for a few minutes, check its status. Most hub implementations provide a health check endpoint or a CLI command to view the current state.
Look for the number of connected peers and the height of the local frame. If the numbers are increasing, your hub is successfully syncing with the mainnet. If not, review the logs for any connection timeouts or authentication errors.
Monitor the multi-channel sync process
Your hub doesn't just download a static copy of the network; it maintains a live, evolving state machine. The sync process relies on a delta-based model, where hubs exchange only the changes (deltas) rather than the entire dataset. This approach keeps bandwidth usage low but requires your node to process these deltas in strict chronological order to avoid conflicts or data corruption.
The sync engine opens multiple parallel channels to different peers. If one pipe delivers water faster than another, the tank's internal mechanism must regulate the flow to ensure the water level (your local database) rises smoothly without overflowing or creating turbulence. In Farcaster terms, this means your hub must handle out-of-order messages and merge them correctly.
When a delta arrives, your hub checks its local timestamp. If the message is older than what you already have, it is discarded. If it is newer, it is applied to your local database. This continuous cycle of fetch, validate, and apply ensures your hub stays in sync with the rest of the network. Monitoring this process involves watching for dropped connections or lagging delta sequences, which can indicate network issues or peer problems.
Because user data changes rapidly, maintaining this sync requires consistent resource allocation. If your hub falls behind, it may need to perform a full snapshot sync to catch up, which is more resource-intensive than incremental delta updates. Keeping an eye on your sync latency helps you identify when such a catch-up is necessary before your hub becomes significantly out of date.
Fix common sync errors
When a Farcaster hub drifts from the network, it usually stems from one of two problems: state divergence or connection timeouts. User data changes rapidly on the network, and if your hub cannot process those changes fast enough, it falls behind. The following steps address the most frequent points of failure.
Resolve state divergence
State divergence occurs when your hub holds a different version of the data than the rest of the network. This often happens after a long outage or a corrupted database. To fix this, you must reset your local state to match the current network consensus.
- Stop the hub service.
- Delete the local database files.
- Restart the hub to begin a full sync from the genesis block or a recent snapshot.
This process is time-consuming but necessary to ensure your data is accurate. Refer to the Farcaster Protocol discussions for details on how ordering affects sync stability.
Handle connection timeouts
Connection timeouts typically indicate that your hub cannot keep up with the incoming message rate or that network latency is too high. If your hub logs repeated timeout errors, check the following:
- Network bandwidth: Ensure your server has sufficient upload and download speed to handle the burst of messages during peak hours.
- Resource limits: Increase the memory allocation for your hub process. Insufficient RAM can cause the sync thread to stall.
- Peer health: Verify that your connected peers are responsive. If a specific peer is consistently timing out, remove it from your configuration to allow your hub to connect to healthier nodes.
Addressing these connectivity issues usually restores normal sync behavior without requiring a full data reset.
Verify your hub is fully synced
Before relying on your hub for production data, you need to confirm it has caught up with the network. A hub that is still syncing may return incomplete or outdated cast data, leading to missed notifications or broken integrations.
Start by checking the latest block height. Compare your hub’s current height against the global network height. You can find the network height using the official Farcaster API or by checking a known healthy hub. If your hub is within a few blocks of the network, it is likely in the final stages of sync.
Next, verify your peer count. A healthy hub maintains connections with multiple other hubs to ensure data redundancy. If your peer count is zero or unusually low, your hub may be isolated and unable to receive updates. Check your hub’s logs for connection errors or firewall blocks.
Finally, test a local query. Request a recent cast or user profile from your hub. If the response is timely and contains the expected data, your hub is ready to serve requests. If the query fails or times out, check your disk space and network connectivity.
-
Check latest block height against network height
-
Verify peer count is greater than zero
-
Test a local query for recent data
Troubleshooting sync issues
When your hub drifts from the network, you usually face three technical hurdles: sync latency, disk space, or peer selection. Here is how to diagnose and fix each one.
Why does sync latency spike?
Farcaster uses a delta-based sync model, meaning hubs compare and copy only new changes rather than rebuilding the entire state. Latency spikes often occur when your hub falls behind the active network. To catch up, ensure your node has a stable connection to multiple peers. The sync process runs in channels, so a backlog in one channel can delay overall consistency. Monitor your hub logs for "catch-up" messages to gauge progress.
How much storage do I need?
Storage requirements grow with the number of messages and casts stored. A fully synced hub requires significant disk space, which can fill up quickly if you do not monitor usage. Check your disk usage regularly and consider using a dedicated SSD for better I/O performance. If you are running a low-resource node, you may need to prune older data or switch to a lighter sync mode.
How do I change my peers?
Peer selection affects sync speed and reliability. If you are syncing slowly, try adding more peers from the official Farcaster network list. You can configure your hub to connect to specific peers by updating your configuration file. Ensure your peers are also fully synced to avoid receiving invalid or outdated data. Restarting your hub after changing peers can help establish new connections faster.
No comments yet. Be the first to share your thoughts!