Infinite Scrolling

Infinite scrolling is a software design technique that loads content continuously as the user scrolls down the page/screen.

There are a few ways to accomplish infinite scrolling with PowerSync, either by querying data from the local SQLite database, or by lazy-loading or lazy-syncing data from your backend.

Here is an overview of the different options with pros and cons:

1) Pre-sync all data and query the local database

PowerSync currently performs well with syncing up to 100,000 rows per client, with plans to scale to over 1,000,000 rows per client soon.

This means that in many cases, you can sync a sufficient amount of data to let a user keep scrolling a list or feed that basically feels "infinite" to them.

Pros	Cons
It works offline and is low-latency (data loads quickly from the local database). We don't need to load data from the backend via the network when the user reaches the bottom of the page/feed/list.	There will be cases where this approach won't work because the total volume of data might become too large for the local database, for example: There's a wide range of tables that the user needs to be able to infinite scroll Your app allows the user to apply filters to the displayed data, which results in fewer pages displayed from a large dataset, and therefore limited scrolling.

2) Sync limited data and then load more data from an API

Here we sync a much smaller number of rows, for example 100 rows. If the user reaches the end of the page/feed/list, we make an online API call to load additional data from the backend to display to the user.

Pros	Cons
This requires syncing less data to each user, which will result in a faster initial sync time.	We can only load additional data when the user is online. There will be some latency to load the additional data (similar to a cloud-first app making API calls) In your app code, records loaded from the API will have to be from the records loaded from the local SQLite database.

3) Client-side triggers a server-side function to flag data to sync

You could add a flag to certain records in your backend database which are used by your Sync Rules to determine which records to sync to specific users. Then your app could make an API call which triggers a function that updates the flags on certain records, causing more records to be synced to the user.

Pros	Cons
This requires syncing less data to each user, which will result in a faster initial sync time.	We can only perform the trigger and sync additional data when the user is online. There will be higher latency: Both for the API call to update the flags, and for syncing the additional data We do not necessarily recommend going this route: There's higher latency and it's not a particularly elegant architecture.

4) [Future support] Control data sync using client parameters

PowerSync currently supports dynamic partial replication by means of token parameters that can be accessed in your Sync Rules (an example of a token parameter is a user ID). Since token parameters are embedded in the authentication token, they can be considered trusted and can be used for access control purposes.

We plan on expanding this and allowing other client parameters to be specified directly by the client (i.e. not only through the authentication token). It will be easier for the developer to dynamically change these parameters on the client-side and access them in Sync Rules on the server-side. The developer can use these parameters to limit/control which data is synced, but since they are not trusted they should not be used for access control. You should still filter data by e.g. user ID for access control purposes (using token parameters as mentioned above).

Usage example: To lazy-load/lazy-sync data for infinite scrolling, you could split your data into 'pages' and use a client parameter to specify which pages to sync to a user.

Pros	Cons
Does not require updating flags in your backend database. Enables client-side control over what data is synced.	We can only sync additional data when the user is online. There will be latency while the user waits for the additional data to sync.

Questions, Comments, Suggestions?

Let us know on Discord.