StableBit CloudDrive 1.X ======================== | Info ==== [D] = Driver was modified Tests ===== (TEST-DC) = Data consistency tests passed across all non-experimental providers (TEST-F) = Functional tests passed (TEST-PF) = Power failure tests passed (TEST-R) = Reboot tests passed (TEST-U) = Upgrade test passed (TEST-V) = Internal system service validation tests passed Summary ======= * Dark mode. RELEASE ======= 1.2.2.1598 ---------- * Release final. BETA ==== 1.2.2.1593 ---------- * BETA. .1592 * Reduced the cloud synchronization frequency on the free tier. * Improved the cloud App UI service startup experience. .1589 * Live connection protocol V10: - Added support for large packets. .1588 * BETA builds now use the BETA cloud API endpoint. * Created a separate app state for locked drives. * Added an update ready cloud app state for notification purposes. .1578 * Added an update available cloud app state. * Enabled cloud synchronization for update settings. 1.2.2.1569 ---------- * Added theme palette support to the cloud dialogs. .1555, .1557 * Disable the remote control bar animation when animations are disabled in the OS. * Added new dark, light, and high contrast themes. .1541 * Detect and mitigate clock shifts into the past when setting cloud states. RELEASE ======= 1.2.1.1539 ---------- * Release final. BETA ==== .1538 * [Issue #28659] Fixed repeated Dropbox reauthorization requests. .1537 * [Issue #28649] Added a ClientSecret parameter to OneDrive for Business in ProviderSettings.json. RELEASE ======= 1.2.0.1534 ---------- * Release final. .1533 * Release candidate. * When connected to the cloud, WMI was causing excessive service startup times on older operating systems. * Added a StableBit Cloud+ trial expiration notice. * [Issue #28642] UI placement persistence was being cleared on non-primary monitors. .1523 * Release candidate. BETA ==== .1520 * When a disk is unmountable, for notification purposes, do not exit the unmountable state until the disk is fully mounted. * Improved HTTP 503 recovery for the StableBit Cloud. * Auto 30 day trial for the StableBit Cloud. 1.2.0.1517 ---------- * BETA. .1516 * [Issue #28635] Fixed written data not being uploaded. .1513 * When linked to the cloud, clicking through to purchase a license from within the app will automatically add the purchased license to the cloud. .1512 * Added support for StableBit Cloud+. .1510 * Fixed an HMAC validation optimization in the live connection protocol. .1509 * If started sending real-time metrics, then keep sending for at least 1 minute. 1.2.0.1508 ---------- * BETA. * Fixed StableBit Cloud integration issues with the Windows Home Server / Essentials. .1502 * Optimized how remote control deals with firewalled peers to reduce CPU usage. Firewalled peers will be rechecked every hour to determine if their firewalled status has changed. .1500 * Fixed an issue in the remote control keep-alive checker that was causing excessive resource consumption when a connection could not be established to a remote IP. .1499 * Faster live connection recovery after connection loss. .1495 * Added support for linking to a site from the StableBit Cloud installer. .1494 * Parallelized pre-shutdown processing. .1493 * Fixed statistics not being sent without a live connection. .1492 * Cloud storable metadata was being persisted to disk too often. .1491 * Only send the real-time metrics that are actively being observed. .1489 * Application settings synchronization updated: - New V4 API (with new back-end store). - Existing settings will be migrated automatically. - Existing links to statistics will be maintained. - Certain features in the cloud will be disabled for older builds that use the previous APIs. .1485 * Send real-time metrics when the web dashboard is open. * Never synchronize unchanged settings to the cloud. .1481 * Fixed a deadlock in the cloud token refresh API. .1478 * Improved the reliability of live reliable packets with a 64-bit sliding window and other improvements. .1477 * Cloud API tier switching improvements. * Handle server keep-alive failures from the cloud API. * A full store synchronization should not take too long when it encounters adverse network conditions. .1469 * Improved how the live connection protocol deals with long-term network interruptions and network changes. .1466 * DataInfo binder should not access disposed objects. * Autofill certain fields when reauthorizing: - Amazon S3. - Backblaze B2. - Google Cloud Storage. - OneDrive for Business. - Rackspace Cloud Files. * Updated the "authorized" page for local OAuth. * Fixed a broken link on the Google Cloud Storage connect dialog. .1465 * Power state transition support for the cloud. * CloudDrive_WaitForInternetS was waiting longer than specified. * Extended the live connection protocol to detect one way restrictive firewalls. .1457 * Optimized code to reduce idle CPU usage. .1456 * Recover from invalid cloud object / change IDs. .1455 * Connect / reauthorize improvements: - Show the connection name when reauthorizing. - Additionally, autofill certain fields when reauthorizing File Share, Local Disk, FTP. - Allow OAuth 2.0 manual connect to accept a full URL for code extraction. - Miscellaneous fixes on multiple connect UIs. .1452 * For the local and file share providers, if directory enumeration fails while rebuilding the chunk ID database, do not continue mounting the drive. * [Issue #28598] Additional optimizations for the chunk ID database. .1450 * [Issue #28594] Fixed excessive memory consumption while rebuilding the chunk ID database. .1449 * Resynchronize app states on live connection loss. * Fixed an incorrect metrics cloud object ID on send. * Moved the data duplication toggle to the data integrity sub-menu. .1446 * Upgraded the chunk ID database access layer. * Live connection protocol updates: - Server connection steering for load balancing support. - Improved connection management. .1444 * Added additional per-disk information to cloud summary app state. .1442 * Added support for StableBit Cloud+ API endpoint switching. * Fetch the live connection servers from DNS. .1439 * [Issue #28571] If an update file is missing when clicking install, requeue it for download. * [Issue #28571] Periodically check to make sure that a downloaded update file is still there. * Added additional live connection servers. * Updated StableBit Cloud dev / local scope domains. .1435 * Fixed restarting the notifications client in a disconnected RDP session. .1432 * Fixed a crash in the activation required notification tray. * Fixed a cloud store synchronization issue that was causing objects to be reverted to their previous state. This was rare but caused all kinds of strange intermittent behavior, such as licenses appearing to get spontaneously deactivated once in a while. * Tweaked cloud license activation timing. * Fixed auto cloud linking. .1424 * When cloud licensing is enabled and there is no license found, explicitly show that in the cloud status dialog. * Updated the cloud update signature keys to be in sync with the StableBit Cloud Installer. .1421 * Integration with the StableBit Cloud Installer (https://stablebit.cloud/Area/Apps): - Do not auto-launch. - Automatically link to the cloud. .1419 * [Issue #28540] Fixed compatibility with iDrive Cloud, see https://stablebit.com/Admin/IssueAnalysis/28540. * [Issue #28540] Added provider settings for Amazon S3: "UseChunkEncoding", "PutBucketUseClientRegion". * [Issue #28540] Updated to the latest Amazon S3 SDK. .1418 * General reliability improvements for AppStates / Notifications. .1414 * Handle unknown storable types during cloud synchronization. .1409 * [Issue #28535] Added a ProviderNaming_RootName advanced setting to control the root folder name at the storage provider. - Use only alphanumeric characters and spaces. - Only for applicable providers: - Backblaze B2 - Google Drive - Microsoft OneDrive (and for Business) - Rackspace Cloud Files .1408 * Allow mount retrying from the cloud. * Bind the drive mounting status to the cloud. .1406 * [Issue #28529] Added options to specify an authentication region and a service name for Amazon S3 third party providers. * Fixed repetitive license queries. .1405 * Report service status to the cloud. * [Issue #28521] Updated the Amazon S3 tooltip to explicitly state that it supports other S3 compatible providers. .1401 * [Issue #28514] Repair the Google Drive METADATA if the content folder was already upgraded to the hierarchical pure format. 1.2.0.1400 ---------- * Metrics / statistics from older builds will no longer be accepted. .1399 * Improved AppState retry logic. * [Issue #28501] Simplified license expiration detection. .1394 * Tweaked the cloud update download retry delay. * Added StableBit Cloud tooltip icons throughout the UI to indicate what is being synchronized with the cloud and in some cases with other apps. * Updated StableBit Cloud branding. .1392 * [Issue #28492] Increased Box's maximum chunk size to 200MB. .1390 * [Issue #28482] Fixed I/O error messages for the Local Disk and File Share providers. * Requeue metrics if unable to send using the API. .1388 * Added support for transactional consistency to AppStates. .1386 * Live Connection protocol V8. .1379 * Increased Live Connection retransmission aggressiveness. .1377 * Statistics are now reported every 30 minutes. * Not all statistics were being reported at shutdown. .1375 * [Issue #28472] Added statistics for the number of API requests made. .1374 * [Issue #28475] Fixed license activation under non-Gregorian calendars. .1373 * [D] [Issue #28469] Fixed formatting as ReFS (broken in .1346). .1362 * Extended the live connection protocol to support health and activity reporting: - Health alerts will be reported at all times. - Activity alerts will only be reported when the dashboard is open. - Summary reporting will be accelerated when the dashboard is open. - The HTTP API will be used when the live connection is not available or as a fallback. .1361 * Restart tray notifications on failure. * Health / activity reporting retry control. .1359 * Enabled real-time metrics in activity alerts. .1358 * Added a health summary to health and activity reporting. .1356 * [D] [Issue #28464] Fixed a race condition handling read / write SRBs right after a cloud drive unmounts. .1354 * Fixed disappearing health alerts after StoreSynchronizationService reconnect. .1352 * Added support for hiding health alerts in the cloud. .1347 * Added the StableBit Cloud health and activity reporting feature. .1346 * Read-only is also now available as an advanced setting in the attach dialog. * The read-only drive state will now persists after reattaching a cloud drive. * [D] [Issue #28456] Report no seek penalty and TRIM enabled on the disk. .1343 * Fixed OS name reporting to the StableBit Cloud. * Fixed cloud licenses not activating in a timely manner. .1339 * Extended the licensing system to be better at automatic activation / deactivation when connected to the StableBit Cloud. .1322 * [Issue #28442] Added the logic from issue #28420 to OneDrive consumer. .1320 * [Issue #28440] Added higher limits to the upload threshold setting. .1317 * [Issue #28420] Treat a 429 response with X-ClientErrorCode: Forbidden from OneDrive for Business as a security error that requires reauthorization. .1316 * [Issue #28419] Fixed drive resizing. .1315 * Fixed a deadlock in the DataInfoBinder. .1314 * [Issue #28415] Created a new chunk organization for Google Drive called Hierarchical Pure. - All new drives will be Hierarchical Pure. - Flat upgraded drives will be Hierarchical, which is now a hybrid Flat / Hierarchical mode. - Upgrading from Flat -> Hierarchical is very fast and involves no file moves. * Tweaked Web UI object synchronization throttling rules. .1312 * Added the drive status bar to the Web UI. .1310 * Tuned statistics reporting intervals to enable additional statistics in the StableBit Cloud. .1307 * Added detailed logging to the Google Drive migration process that is enabled by default. * Redesigned the Google Drive migration process to be quicker in most cases: - For drives that have not run into the 500,000 files per folder limit, the upgrade will be nearly instantaneous. - Is able to resume from where the old migration left off. * [Issue #28410] Output a descriptive Warning to the log when a storage provider's data organization upgrade fails. .1306 * [Issue #28408] Reduced GoogleDrive_ConcurrentRequestCount to 2. .1305 * Added a progress indicator when performing drive upgrades. * [Issue #28394] Implemented a migration process for Google Drive cloud drives to hierarchical chunk organization: - Large drives with > 490,000 chunks will be automatically migrated. - Can be disabled by setting GoogleDrive_UpgradeChunkOrganizationForLargeDrives to false. - All drives can be migrated by setting GoogleDrive_ForceUpgradeChunkOrganization to true. - The number of concurrent requests to use when migrating can be set with GoogleDrive_ConcurrentRequestCount (defaults to 10). - Migration can be interrupted (e.g. system shutdown) and will resume from where it left off on the next mount. - Once a drive is migrated (or in progress), an older version of StableBit CloudDrive cannot be used to access it. * [Issue #28394] All new Google Drive cloud drives will use hierarchical chunk organization with a limit of no more than 100,000 children per folder. .1301 * Fixed bandwidth reporting accuracy for the StableBit Cloud. * Web UI compatibility changes for the StableBit Cloud. * Fixed metrics flush on shutdown. .1296 * Added per-object type synchronization throttling rules. * Real-time object synchronization is now active only when the StableBit Cloud Web UI is open. - Reported in the live connection status tooltip. .1292 * Improved StableBit Cloud uptime status reporting: - Before allowing new connections, check the availability of the service. - If the service becomes unavailable while connected, all functionality pauses. - Do not show security specific errors unless there is an explicit deauthorization from the service. .1291 * [Issue #28321] Simplified the Backblaze B2 login UI. .1290 * Fixes for StableBit Cloud license management. .1287 * Added status tooltips to each connected StableBit Cloud service: - Application settings synchronization: - Shows how many objects were synchronized to / from the cloud in this session (objects are only synchronized when they change). - Live connection: - Shows the current connected host. - Packet loss percentage on send / receive. - Packet retransmission factor on send (e.g. x1.0 means 1 additional packet had to be sent to ensure reliability of the connection per each one sent normally). - License management: - Shows whether cloud licensing is enabled. - If enabled, shows whether a license was found in the cloud or not for the current app. - Enhanced update delivery: - Shows the currently subscribed to scope and channel for update push notifications. - Shows if an update was found and which version. - Metrics and statistics: - Shows if real-time metrics are currently being sent via the live connection (if the Web UI is open in the cloud). .1286 * Lowered timeouts on metrics API calls. * Fixed a thread pool exhaustion caused by metrics if the server is not responding. .1282 * [Issue #28313] If a cloud drive is not marked as attached at the time of drive mounting, mark it as attached and continue the mount. * [Issue #28313] If a cloud drive was force attached to a different computer while the original computer was offline, and then subsequently the drive was detached from the other computer, the original computer would then get an "Object reference not set to an instance of an object" error when mounting that cloud drive. .1280 * Improved overall handling of StableBit Cloud server downtime. .1274 * [Issue #28312] Added additional details when logging I/O manager mounting errors. * [Issue #28310] Added additional details when logging FTP errors. .1273 * Live connection protocol reliability fixes. * Fixed a lockup in the StableBit Cloud token service. * Added a backup automatic update mechanism. * Metrics improvements to prevent double reporting when the transmission protocol switches. .1266 * [Issues #28306/28307] Fixed error reporting. .1261 * [Issue #28303] Added more meaningful error messages when there are possible nulls. .1260 * Added metrics support for the StableBit Cloud: - Various metrics are supported, depending on the application (e.g. disk space, performance, bandwidth). - Metrics can be sent either in real-time (for immediate display purposes), or periodically (for historical statistics). - Real-time metrics require a live connection and will be enabled as needed, depending on the web app usage. - Historical metrics are sent at all times. - 3 transmission protocols are supported: - Unreliable secure UDP (very low overhead): used for real-time metrics. - Reliable secure UDP (low overhead): used for historical metrics. - JSON HTTP (normal overhead): used for historical metrics when UDP is not available or cannot be used due to poor network conditions. - Batching of multiple metrics is supported on all protocols for efficient transmission (up to 60 metrics per 1000 bytes for UDP). .1248 * Updated the Google Cloud Storage API keys. .1245 * Google Drive can now accept additional users and is no longer marked as experimental. .1243 * Added new conversions to CloudDrive.Convert.exe: - [Issue #28262] DropboxToLocalDisk. - LocalDiskToFileShare. - FileShareToLocalDisk. .1242 * Fixed various issues with the Login UIs. * Clean up invalid settings. * Do not synchronize task ended user feedback to the cloud. .1232 * Google Drive is now marked as experimental. - The app is approaching the queries per 100 seconds limit set by Google. - Google has not approved our request to increase this limit at this time. * The Google Drive API key is now verified by Google. * If remote control cannot be started for any reason, show a warning in user feedback. .1228 * [D] Fixed a rare race condition causing a PAGE_FAULT_IN_NONPAGED_AREA in write path timeout. * [Issue #28259] Optimized drive cleanup database access for very large drives. * Added a Null provider to the DEV BETA for testing purposes. .1223 * Added the cloud DataInfo binder, enabling the ability to build real-time app UIs in the cloud. .1220 * Updated the remote control protocol to handle computer names more consistently. .1219 * For remote control, fixed multicast when multiple connected adapters are present and the LAN adapter doesn't have the lowest interface metric. * [Issue #28253] Add a what's new link to the automatic cloud update dialog. .1215 * The service will now start and stop the notification UI automatically for all active sessions. .1211 * Improved the reliability of the live connection in the presence of heavy packet loss. * Added StableBit Cloud powered automatic updates. - Instant update notifications. No need to "check for updates", updates are pushed to you in real-time as soon as they're released. - Support for multiple release channels (Dev BETA, BETA, Release Final). - Automatic downloads. - Legacy automatic updates will be disabled when connected to the StableBit Cloud. .1189 .1187 * Added centralized licensing with the StableBit Cloud. - The local licensing UI will be inaccessible when using cloud licensing. * Added dev / local server toggles for the StableBit Cloud. * Added the ability to target specific versions with automatic updates. .1184 * Enabled StableBit Cloud integration: - Application settings synchronization: - Synchronizes all non-security related persistent settings to the cloud. - If something goes wrong and you lose your settings locally, or you have to reinstall the OS, simply connect the app to the cloud to restore your settings. - StableBit CloudDrive specific features: - Any security related information is never sent to the cloud (e.g. storage provider authorizations, encryption keys / passwords). You are responsible for maintaining those locally as before. - Default I/O performance settings are synchronized across all installations at a particular site, in real-time. - Per drive persistent data (such as I/O performance, pinning settings, read-only state, etc...) are automatically synchronized across different app instances connected to the same StableBit Cloud account. * Fixed Rackspace Cloud Files user manual link. Release ======= 1.1.7.1391 ---------- * [Issue #28482] Fixed I/O error messages for the Local Disk and File Share providers. * [Issue #28475] Fixed license activation under non-Gregorian calendars. * [D] [Issue #28464] Fixed a race condition handling read / write SRBs right after a cloud drive unmounts. * [D] [Issue #28456] Report no seek penalty and TRIM enabled on the disk. * [Issue #28415] Created a new chunk organization for Google Drive called Hierarchical Pure. - All new drives will be Hierarchical Pure. - Flat upgraded drives will be Hierarchical, which is now a hybrid Flat / Hierarchical mode. - Upgrading from Flat -> Hierarchical is very fast and involves no file moves. * [Issue #28442] Added the logic from issue #28420 to OneDrive consumer. * [Issue #28420] Treat a 429 response with X-ClientErrorCode: Forbidden from OneDrive for Business as a security error that requires reauthorization. 1.1.6.1318 ---------- (TEST-U) * [Issue #28436] Fixed ReFS pinning. 1.1.5.1249 ---------- (TEST-U) * [Issue #28283] Updated the Google Cloud Storage API keys. 1.1.4.1244 ---------- (TEST-U) * Google Drive can now accept additional users and is no longer marked as experimental. 1.1.3.1221 ---------- (TEST-U) * Google Drive is now marked as experimental. - The app is approaching the queries per 100 seconds limit set by Google. - Google has not approved our request to increase this limit at this time. * [D] Fixed a rare race condition causing a PAGE_FAULT_IN_NONPAGED_AREA in write path timeout. * [Issue #28259] Optimized drive cleanup database access for very large drives. * The Google Drive API key is now verified by Google. 1.1.2.1178 ---------- (TEST-DC) (TEST-F) (TEST-PF) (TEST-R) (TEST-U) (TEST-V) * Release final. * Fixed excessive memory consumption due to item change notifications. RC == 1.1.2.1177 ---------- (TEST-F) * Release candidate. * Fixed a lockup in the entropy collector which was causing the UI to freeze when creating an encrypted cloud drive. BETA ==== .1174 (TEST-DC) (TEST-PF) (TEST-R) (TEST-U) (TEST-V) * Optimized how Store item change notifications are issued: - Coalesce multiple item change notifications into a single notification if they are issued over the same object and happen within the same lock context. - Use 2 dedicated background item change notification threads that are separate from other change notification threads. - Do not throttle item change notifications if the maximum notification queue size is exceeded, simply drop any additional notifications and log a warning. .1173 * [D] Fixed spontaneous cache trimming. 1.1.2.1171 ---------- (TEST-DC) (TEST-F) (TEST-U) (TEST-V) * [Issue #28212] Fixed excessive CPU usage on upload (especially when dealing with larger cloud drives). (TEST-PF) (TEST-R) .1170 * The Amazon Drive provider is now hidden. - Amazon is suspending StableBit's API keys on November 8, 2019. No reason was given. - From Amazon: "We apologize for the impact this has on your application, and thank you for being a developer on our platform." .1169 * When a cloud drive couldn't be mounted due to the storage provider being inaccessible, especially when auto mount retry was enabled, a resource leak was possible causing excessive CPU and memory usage leading to general instability. .1168 * Added ReFS support to filesystem metadata and directory index pinning. - Contains a full independent filesystem reader based on reverse engineering. - Supports ReFS versions: - 1.1 (Windows Server 2012) - 1.2 (Windows Server 2012 R2) - 3.1 (Windows Server 2016) - 3.4 (Windows Server 2019 / Windows 10) - This also adds ReFS support to pinned data duplication. Release ======= 1.1.1.1165 ---------- (TEST-PF) (TEST-R) (TEST-U) (TEST-V) * Release final. * Release candidate. BETA ==== * Fixed "Chunk ID was not set" error in the new indexed providers. * Fixed OpenDrive cleanup. * Properly support down-level logon name format user names (DOMAIN\USER) for file shares. * Do not offer ReFS as an option on non-server Operating Systems (except Windows 10 Pro Workstation and Enterprise). 1.1.1.1159 ---------- (TEST-F) (TEST-U) * Fixed serialization of the new verify data task state. * Enabled ReadModifyWritePartialChunkRecoveryImplementation_EnablePartialChunkRecovery. * Fixed RMWR exception for unexpected data blocks. .1153 (TEST-DC) (TEST-PF) (TEST-R) (TEST-V) * Temporary writes cannot be allowed to write past the end of the drive. - This was causing the cache to get into a state which prevented some data from being trimmed (at the end of the disk). Detaching the drive and reattaching it gets it out of that state. * [D] Added pinned data duplication. (Manage Drive -> Data duplication... -> Duplicate pinned data only) - If enabled, when data is written to the storage provider, if that data intersects with a pinned region then it will be written twice in order to duplicate it for redundancy. - When cleanup runs, duplicated chunks that no longer intersect pinned regions will be automatically cleaned up. - If pinned data duplication is enabled, cleanup will run automatically every IoManager_AutomaticCleanupInterval (7 days by default). - If pinned data duplication or partial drive duplication are enabled, then pinning will ensure that all pinned data is duplicated every PinDiskMetadata_IntervalM minutes (60 minutes by default). * [D] Fixed SCSIOP_VERIFY16 (chkdsk uses this when checking for bad sectors). * For new drives using the local disk or file share providers, duplicate chunk parts will now be prefixed with the part number for better compatibility with StableBit DrivePool's file placement rules. * Fixed cleanup for Microsoft Azure and Google Cloud Storage. * New drives created with all storage providers are now indexed, which includes these previously unindexed providers: - Dropbox - File Share - FTP - Google Cloud Storage - Local Disk - Microsoft Azure - OneDrive - OneDrive for Business * When creating a new drive, chunk validation is now forced on when data duplication or encryption are enabled. * [D] Data duplication reliability improvements: - Data duplication can no longer be enabled on non-indexed storage providers. (there's just not enough data integrity information there to provide a useful benefit). - Read-modify-writes can now recover from failures using duplicate chunks. - Upon encountering missing or damaged data, a healing process will automatically regenerate the damaged data from known good duplicate parts, in real-time, while the drive is online. A UI feedback message will be posted when this happens. Partial chunk recovery and full chunk recovery are supported (ReadModifyWritePartialChunkRecoveryImplementation_EnablePartialChunkRecovery). .1145 * [D] [Issue #28137] When the minimum download size was enabled, data at the very end of the disk could not be read. .1131 * [D] [Issue #28143] Fixed a page fault in write I/O. .1129 * Display the drive name and UID when prompting for drive encryption credentials. * Print / save the drive UID when backing up the drive encryption credentials. * Updated some icons / colors in the UI. * [Issue #28140] Changed how data speeds are displayed: - Bytes per second (decimal): Bytes/s, kBytes/s, mBytes/s, etc... - Bytes per second (binary): Bytes/s, KBytes/s, MBytes/s, etc... - Bits per second (decimal): bits/s, kbits/s, mbits/s, etc... - Bits per second (binary): bits/s, Kbits/s, Mbits/s, etc... * Fixed and optimized block hash storage for improved chunk validation when partial data duplication is enabled. 1.1.1.1128 ---------- (TEST-U) (TEST-F) * Public BETA build. .1126 (TEST-DC) (TEST-V) (TEST-PF) (TEST-R) * Fixed validation after hitting the default action button on the connect dialog. * Added a "Caution" message for Google Drive users about the June 2 2019 Google services outage. .1124 * Fixed a rare deadlock in the ThreadPool having to do with setting the maximum number of threads and thread aborts. * [D] [Issue #28134] Cap the increase of the size of the flat skip list. * [D] [Issue #28134] Handle flat skip list reallocation failures more gracefully. * [D] Fixed reconnecting a cloud drive that was mounted in offline mode. .1121 * Added an option to enable or disable data duplication for existing cloud drives (Manage Drive -> Data duplication...). - Any new data written to the cloud drive after duplication was enabled will be stored twice at the storage provider. - Existing data on the drive that is not overwritten will continue to be stored once. .1118 * Added an option to enable data duplication when creating a new cloud drive. - Data duplication stores your data twice at the storage provider. - It consumes twice the upload bandwidth and twice the storage space at the provider. - In case of data corruption or loss of the primary data blocks, the secondary blocks will be used to provide redundancy for read operations. .1114 * Improved chunk validation. - For indexed providers, if a valid but unexpected block (that was cryptographically signed by us) comes back from a storage provider, do not accept the data and issue a warning stating that the data was corrupted at the storage provider. - When this happens, the block will be retried 3 times and if the problem continues your cloud drive will be force unmounted. - This prevents us from accepting data that is stale or out of order. * Improved UI startup time. .1106 * Fixed user manual link to Backblaze B2. * [Issue #28054] Fixed the use of legacy master keys in Backblaze B2. * [Issue #28102] New cloud drives created using the FTP storage provider will now organize their chunks hierarchically. - Existing cloud drives can be upgraded to the hierarchical format by setting Ftp_UpgradeChunkOrganization to true and rebooting (or detaching and reattaching the cloud drive). Upgrading existing drives is disabled by default. * [Issue #28065] Do not allow a drive to mount on service start if it was force attached to a different system while the computer was shut down. - Note that attaching the same cloud drive to more than one computer at a time is still not allowed. .1065 * [Issue #28053] Changed Mbps -> Mbits /s in I/O performance for clarity. .1064 * [Issue #28054] Added support for standard Application Keys to Backblaze B2. - You will need to enter an additional "Application Key ID" when using a non-master key. - Master keys will continue to work as before, simply omit entering the application key ID when connecting. * [Issue #28054] Updated Backblaze B2 to use the latest v2 API. .1063 * Updated dotConnect for SQLite to 5.11. * [D] [S] [Issue #28047] Factored out ExRaiseStatus from read / write dispatch routines. 1.1.1.1060 ---------- (TEST-DC) (TEST-V) (TEST-R) (TEST-PF) (TEST-F) (TEST-U) * [D] [S] Rebuilt with 17763.134 WDK. Tested and signed for Windows Server 2019. .1058 * Made the licensing engine FIPS compliant. .1057 * Do not save a copy of the most recent log entries along with an error report (ErrorReporting_SaveLog). .1056 * [Issue #27968] Added a concurrency limit to metadata pinning (PinDiskMetadata_Concurrency). .1055 * [Issue #27954] Updated the Amazon S3 SDK to the latest. .1054 * [Issue #27935] Fixed an error while mounting cloud drives that are using large chunk sizes (> 2GB). .1053 * Use fixed sized (1 MB) buffers for local drive / file share providers. * [Issue #27912] When attaching an existing cloud drive, the cache can be > 500 GB. Release ======= 1.1.0.1051 ---------- (TEST-R) (TEST-F) (TEST-U) * Release final. * Hide testing options in Technical Details for release builds. * In the UI, show the drive mount status earlier at service startup. * OpenDrive changes: - Marked OpenDrive as experimental due to multiple server issues encountered. - Disable HTTP compression (server sends invalid blocks sometimes). - Workaround for broken HTTP GET range requests. - Added "PartialReads" provider setting (defaults to true). Setting this to false will disable HTTP GET range requests. - Workaround for 25000 file limit per directory. - Existing drives using the flat layout scheme will be automatically upgraded to the recursive scheme on first mount. - The conversion process may take some time for large drives. * Fixed popup placement (workaround for bug in .NET). RC == 1.1.0.1038 ---------- (TEST-U) (TEST-F) * [D] [S] Fixed a crash on shutdown. .1036 (TEST-DC) (TEST-V) (TEST-PF) (TEST-R) * Release candidate. BETA ==== .1035 * Use separate cache handle groups for reads vs. writes. - Reduces read / write thread contention. - Improves read performance when background I/O is enabled on upload threads. .1034 * Added advanced setting to not use Background I/O when writing to the storage provider, only use it when reading from the local cache. - Can be enabled by setting IoManager_NoBackgroundIoOnProviderWrite to true. - Affects Local Disk and File Share providers. * [D] [S] Changed cache block deprioritization rules to never deprioritize existing data, only new data. - Improves cache utilization when the minimum download size is enabled. .1031 * [Issue #27850] Added additional FTP / SFTP advanced settings to %PROGRAMDATA%\StableBit CloudDrive\ProviderSettings.json. - Each setting contains a description of what it does and the expected value type. - Many of these enable workarounds for issues with specific FTP servers. If you're experiencing problems with FTP / SFTP, toggling these settings may provide for a workaround. .1030 * [Issue #27864] Always turn off "Stop uploading on..." options on service start (unless debugging). 1.1.0.1029 ---------- (TEST-U) * [D] [S] [Issue #27859] Implemented unmount priorities for cloud drives. - At pre-shutdown, a graph of all pools, pool parts, remote and local cloud drives will be built. - Each cloud drive will be assigned an unmount order so as not to affect the other cloud drives when unmounting. * [Issue #27859] Fixed an issue with the Local Disk provider storing data on a StableBit DrivePool pool, when the pool consists of at least one other cloud drive. .1026 * If while pinning, some data cannot be made resident, continue pinning. 1.1.0.1025 ---------- (TEST-U) (TEST-F) * Updated CRT dependency to the latest 2017 and fixed deployment on x86. .1024 (TEST-DC) * [Issue #27758] Fixed an issue computing the minimum recommended download speed. - This was preventing the I/O performance window from opening when download throttling was enabled. * [D] [S] [Issue #27816] Do not prefetch data that has already been cached locally. .1021 * Do not preallocate chunk data when using the Local Disk provider. .1020 (TEST-V) (TEST-R) (TEST-PF) * Added default I/O performance settings. - Enabled default settings will be applied to all newly created cloud drives. - Settings that exceed limits defined by the storage provider, or usability limits, will be automatically constrained. .1017 * Added warning messages if the I/O performance window cannot find the required Store objects in memory. .1016 * [Issue #27849] Fixed connecting with the FTP provider. * Added Http_UserAgent advanced setting to configure the User-Agent that is sent to HTTP servers. - {0}: Application version. - {1}: OS version. - Can be cleared to avoid sending the User-Agent. * Tweaked the connection UI for Microsoft Azure Storage. * [Issue #17732] Added "OpenDrive" storage provider: https://www.opendrive.com - Upload verification is enabled by default. * [Issue #27815] Updated the Rebex FTP component to the latest version (2018 R1.1). .1008 * [Issue #27831,#27832] Fixed "item does not exist" error when connecting to OneDrive. * When running the incorrect installer architecture (x86 vs x64), display a more informative error message. .1007 * [Issue #27837] Fixed driver installation / upgrade issues on Windows 6.3 and Windows 10.0. .1003 * Amazon S3 is now an indexed storage provider. * [Issue #26034] Added support for 3rd party Amazon S3 compatible service providers. - When connecting to Amazon S3, you can now choose to use an Amazon S3 region or enter a custom service URL. - Selecting an Amazon S3 region when creating a new drive is no longer necessary. Region selection is now per connection. Older drives with regions selected will continue to work as before. - For custom service URLs, new advanced provider settings are available (%PROGRAMDATA%\StableBit CloudDrive\ProviderSettings.json): - ForcePathStyle (bool): [true] Force path style API calls. - SignatureVersion (string): [null] Can be set to "2" for legacy servers, defaults to auto. - SignatureMethod (string): [null] Can be set to "HmacSHA1" or "HmacSHA256", defaults to auto. * Compiled using Visual Studio 2017. * Updated dotConnect for SQLite to 5.9. .998 * [Issue #25977] Added Rackspace Cloud Files storage provider. .994 * Fixed drive mounting after unsafe start. .991 * [Issue #20801] Added Backblaze B2 storage provider. .989 * Fixed an access denied error when disabling read-only mode with open files on the drive. - Added an option to force close existing open files and force disable read-only. .988 * Added an option to automatically retry mounting drives. See "Automatically retry mounting" under Options. - Drives that were unmounted due to an incorrect or missing encryption key, an unknown storage provider, or damaged metadata will not retry mounting automatically. .986 * [S] Windows 10 / Server 2016 driver is now approved and signed by Microsoft. * [D] Driver code signing now uses the Covecube Inc. EV (extended validation) certificate. * [D] Tweaked the cache learning algorithm to be more aggressive at keeping the most recently accessed data. .982 * [Issue #27765] Added FileShare_LegacySecurityContext advanced setting to revert the file share provider back to using the old authentication method. .981 * Fixed upgrade problem with installer. * [D] Authenticode signing now uses the Covecube Inc. EV (extended validation) code signing certificate. * Changed CRT dependency to 2017 (v141). Release ======= 1.0.2.976 --------- (TEST-U) * Release final. * [Issue #27749] Fixed OAuth 2 token refresh for Google Drive. RC == 1.0.2.975 --------- (TEST-U) (TEST-F) * Added IoManager_ReadThreadPriority setting to control the priority of read threads. - Defaults to THREAD_PRIORITY_HIGHEST (2) - Possible options: https://msdn.microsoft.com/en-us/library/windows/desktop/ms686277(v=vs.85).aspx .974 (TEST-U) * Fixed mounting speed issue when cloud drives are part of a StableBit DrivePool (.891+) pool. .973 (TEST-DC) * Some provider write I/O errors were being retried perpetually and were never reported in user feedback. * Connection limiting is now part of the chunk I/O pipeline and doesn't depend on the provider API (for providers such as FTP). .972 (TEST-V) (TEST-R) (TEST-PF) * [D] Release candidate. BETA ==== .971 * [D] Improved the reliability of the file share provider. .967 (TEST-DC) (TEST-V) (TEST-PF) (TEST-R) * [D] Potential release candidate. .965 * [D] Unpause uploads when the write map is full (for fixed / proportional cache). .964 * Fixed boot time logging. * [Issue #27670] Added Http_Legacy10 advanced setting. When set to true, the HTTP protocol will use 1.0 instead of 1.1 throughout the application. * [D] Reading encrypted data from a cloud drive would sometimes generate I/O errors. 1.0.2.963 --------- (TEST-F) (TEST-U) * [D] Fixed a race condition where there was a tiny chance that a read would never finish when the minimum download size was enabled. .962 * [Issue #27528] Downgraded common log warnings due to Google Drive throttling to Informational level. .961 (TEST-R) (TEST-DC) (TEST-V) (TEST-PF) * Fixed deadlock in reading JSON settings. * Updated the Rebex FTP component to the latest version (2017 R4.1). .958 * [Issue #27687] Added IoManager_NoFreeSpaceCheckOnStorageProvider advanced setting to bypass the free space check at the storage provider when creating new cloud drives or resizing existing ones. .957 * [Issue #27541] Fixed handling HTTP throttling responses (429) from OneDrive for Business. .953 * Fixed rare performance sampling error. .952 * [D] [Issue #27673] Fixed crash in write flush code when drive was unmounting. .951 * Fixed a service crash on mount. * Attaching an encrypted drive didn't select it by default. * [Issue #27676] Fixed unlocking an encrypted drive. .950 * [Issue #27632] When creating / attaching drives, if there are no suitable destination volumes available for the local cache, show an error message instead of crashing. .949 * [D] [Issue #27641] I/O queues were sometimes not being purged when a cloud drive was unmounted. * Added options to simulate read / write timeouts and complete read lockups to the technical details window (for testing purposes). - Read / write timeouts simulate the storage provider not responding in a timely manner. - Read lockups simulate a complete service lockup (e.g. due to an I/O deadlock). The kernel driver will be forced to recover by forcefully unmounting the cloud drive in about 6 minutes. .948 * [Issue #27643] Do not retry mounting the same cloud drive if the previous mount had failed for any reason, unless the user clicks "Retry...". .947 * [D] [Issue #27657] Added an option to the troubleshooting menu to manually force startup recovery. - This is the same recovery that takes place after a power loss. * Unsafe start user feedback was not always shown due to drive mount timing. * [D] [Issue #27657] 0 length ranges were allowed. - If read / write / pin maps contain invalid ranges, the maps will automatically recover using unsafe start recovery on the next mount. * Fixed error saving JSON service settings to Store. .945 (TEST-U) .944 (TEST-F) * Preserve the currently selected drive across UI restarts. .943 (TEST-DC) (TEST-PF) * Fixed an error assigning a drive letter. * Fixed an issue where sometimes more upload threads were allowed to run than specified. This was causing issues for FTP connection limiting. * It was not possible to turn off "maximum connections" in I/O Performance. .940 (TEST-V) (TEST-R) * Never allow the service to launch multiple times. .939 * Fixed issues applying JSON service settings. .938 * Fixed drive selection bar not loading when there are no cloud drives. .937 * Added the ability to pause all uploads on all cloud drives from the global Settings menu. * Added an upload pause toggle button to the bandwidth indicator in the UI. Uploads will automatically resume if the cache drive starts running low on disk space. - Existing uploads will run to completion. * The minimum allowed download throttling speed is now computed using the capabilities of the storage provider as well as various application settings. * Added the minimum recommended download speed to the I/O Performance window. - This is computed based on the capabilities of the storage provider and various application settings. Changing the number of download threads as well as the minimum download size will recompute this recommendation. .935 * Show the maximize / minimize buttons in the vertical UI. * [Issue #27544] Canceling an update download wouldn't work in a timely manner if the server wasn't responding. .934 * Drive letter assignment is now mount folder aware. .933 * [D] Fixed error setting drive to read-only when it had no volumes on it. * Removed VDS refresh after every mount. It was slowing things down. * Added the ability to easily assign and remove drive letters from the Manage Drive menu. - The same drive letter will be reassigned to the same drive, whenever possible. * Fixed DPI on PNGs. .930 * [Issue #27635] Fixed chunk indexing getting reset. * Reduced CloudDrives_MountParallelization to 2. * Added a context menu to the drive selection bar. 1.0.2.929 --------- (TEST-U) * Fixed service crash when Settings.json doesn't exist. .925 * Fixed thread abort mechanism for write (upload) threads. Threads were being aborted prematurely. * TimeSpan typed Settings.json overrides were being ignored. .919 * Set keep-alive timeout to 10s (was 100s). .918 * Fixed pre-release banner visibility. .917 * Increased the IoManager_HttpConnectionWriteTimeoutMS to 3600000 (1 hour). .916 (TEST-F) * Fixed cache size tooltip. .915 * List Google Drive files 1000 at a time (instead of 100 at a time). .914 (TEST-DC) (TEST-R) (TEST-PF) (TEST-V) * [Issue #27563] Add retry control to the file enumeration process on Google Drive. * Added additional synchronization with StableBit DrivePool .834+ on service start. .913 * [Issue #27612] Allow the cache to be stored on a SD bus (for eMMC compatibility). .912 * Remember and restore the main window's startup position. .910 * [Issue #27614] Sanitize sensitive data from task states before persisting them to disk. .909 * [D] Fixed opt-in for no-execute (NX) non-paged pool. .908 * [Issue #27613] Fixed "Wrong Type" error on Windows 7. * [Issue #27610] Implemented local error reporting for the UI. Error reports will be saved to: %PROGRAMDATA%\StableBit CloudDrive\UI\ErrorReports .906 * Added CloudDrive_SynchronousServiceStart setting (false by default). When enabled, the windows service will not go into the started state until all cloud drive volumes are ready. * Wait for all of the cloud drives to finish mounting their volumes on service start. .904 * Advanced application settings are now stored in %PROGRAMDATA%\StableBit CloudDrive\Service\Settings.json: - Each setting has a "Default" and "Override" values. - When "Override" is set to null, the "Default" value will apply. - Set "Override" to some value other than null in order to override the default. - Setting changes are applied instantly, but some settings may require a service restart or a reboot in order to go into effect. .903 * [Issue #27578] Disk options were not being applied to the I/O manager in real-time. .902 * [D] [Issue #27583] Fixed a race condition if force unmounting a cloud drive while writing to it at the same time. .901 * Wait for cloud drives to get a disk number before ending async service start. * [D] Parallelized and optimized drive mounting on initial service start. * Turning off throttling would not work. .900 * Added coordination between StableBit CloudDrive (.900 and newer) and StableBit DrivePool (.798 and newer) in order to ensure that any cloud drives part of the pool don't cause missing disk warnings, incomplete file warnings, unnecessary remeasures, and unnecessary duplication passes after a reboot. .898 * Move BitLocker monitoring initialization into async start block (it takes too long to load at boot time). .897 * Default to a 1 MB minimum download size for most providers. .896 * Retry for 3s when encountering "The remote procedure call failed and did not execute" (RPC_S_CALL_FAILED_DNE) for the file share provider. .895 * [D] Prefetcher was not always being triggered with the new minimum download size setting. .894 * Do not abort write requests that are waiting for data to be pre-buffered. .893 * In addition to pinging an IP, resolve a host name (CloudDrive_ResolveHost) before starting to mount cloud drives. .888 * [D] Overlapping provider read requests will now be attached to existing ongoing reads in the kernel, instead of generating separate requests. The number of attached vs. regular requests can be observed in Technical Details when hovering over the Read or Prefetch nodes. * [D] Redesigned how the minimum download size works: - It is no longer a provider metadata option, and it can no longer be set at drive creation / attach time. - Use the I/O Performance window to change the minimum download size at any time. - The minimum download size now works exclusively in the kernel and not in the service. Read requests will respect the minimum download size, unless some of the data in the block is completely unnecessary. - The local cache is now aware of the minimum download size, and it will store excess data with a low priority. If nothing reads that excess data, it will be trimmed first. - The prefetcher remains aware of the minimum download size, as it did previously. * [D] Fixed a service crash on drive unmount. .885 * [D] Fixed IRP DeviceObject target when interfacing with the local cache. .884 * Increased IoManager_HttpConnectionWriteTimeoutMS to 20 minutes. .883 * [Issue #27483] When encountering an invalid filename in the DB directory, do not crash. * [D] ReFS volumes can now serve as the cache. Release ======= 1.0.1.880 --------- * [D] Fixed a crash on driver unload. .879 (TEST-DC) (TEST-V) (TEST-PF) (TEST-R) (TEST-F) (TEST-U) * 1.0.1 release final. BETA ==== * A drive that is part of a StableBit DrivePool pool cannot be made read-only. * [D] Kernel code in common with StableBit DrivePool is now shared with StableBit CloudDrive. .877 * Renamed "Drive Options" to "Manage Drive" and reorganized it. * [D] [Issue #27501] Added "Mount disconnected" to the Troubleshooting menu. - A cloud drive can now mount in a disconnected mode when the storage provider is not accessible. - When mounted in disconnected mode, the cloud drive accesses data from the local cache only and remains read-only. - Disconnected mode is for troubleshooting and data recovery purposes only, as some data on the drive may be missing. * [D] Added read-only mode to cloud drives. - When a cloud drive is in read-only mode, writes to the drive are prevented on the virtual disk. - The read-only mode state of a cloud drive is preserved on the local system. - Entering read-only mode requires all of the volumes on the cloud drive to be unmounted and remounted. If there are open handles to files on the cloud drive, you may get an "Access Denied" error. You can check "Force close existing files" to force read-only mode. .875 * Updated Dropbox to the v2 API. .874 * Updated to WiX 3.11. * Added CloudDrive.Convert.exe to \Tools sub-folder under the installation path. - Able to convert cloud drives from Amazon Drive to Local Disk. Release ======= 1.0.0.870 --------- (TEST-F) (TEST-U) * 1.0.0 release final. RC == * Fixed "Unlock" button not getting disabled when a root task is running. .867 (TEST-DC) (TEST-V) (TEST-R) (TEST-PF) * New drive creation disabled for Amazon Drive. * Minor UI tweaks. .862 * Exposed a cloud drive's creation time and version, if known, under Technical Details. .861 * Moved "Ignore chunk verification" to the Troubleshooting menu. .860 * Fixed OneDrive directory enumeration which was affecting Cleanup. * Allow cluster size selection for ReFS volumes and do not force 64 KB clusters for ReFS. .859 * [Issue #27420] When multiple cloud part folders exist on Google Drive for the same drive, do not mount the drive. Instead, show an error to the user. .858 * [D] [Issue #27416] When the system service stopped responding to I/O requests, as a last resort, the drive was being forced to unmount. But because of a race condition, when this occurred, it was possible that the virtual disk would receive a SSD TRIM request that would crash the system. That SSD TRIM command will now be failed. .857 * [Issue #27406] Lowered Box organizational upgrade parallelization to 2 threads (from 10). .856 * [Issue #27404] WMI monitoring added for BitLocker volume arrival (BitLocker_CloudPartUnlockDetect). .855 * [D] [Issue #27388] When unmounting a cloud drive with no volumes on it, and the system contains one or more volumes that do not respond to IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS (e.g. emulated DVD drives that don't support it, floppy drives, etc...), an error would be incorrectly generated on unmount. 1.0.0.854 --------- (TEST-U) (TEST-F) (TEST-DC) (TEST-V) (TEST-R) (TEST-PF) * Fixed FTP provider cleanup. * Added a pre-release banner. BETA ==== .852 * [D] [Issue #27379] Box cannot list the contents of folders that contain > 300000 files (undocumented API limitation). - As a result of this limit, the chunk file organization has to change in order to avoid ever hitting it. - Newly created drives will use the new organization by default. - Legacy drives that can potentially require > 300000 chunks will be upgraded to the new chunk organization format. - The upgrade is parallelized, but it may take a while because all of the chunks will need to be moved to new folders. - If an upgrade is interrupted (e.g. system shut down), it will resume from where it left off on the next drive mount. - A chunk reenumeration will take place after the organizational upgrade. .850 * [Issue #27375] Never allow write thread boosting to issue more write requests than is specified in upload threads. * Purge read-modify-write data on write completion, detachment, and destruction. .848 * Fixed manual code entry for OAuth. * [Issue #27370] Fixed implicit/explicit FTP connection when specifying a custom port. * Fixed manual OAuth reauthorization when using local web server. 1.0.0.846 --------- * [Issue #27364] Chunk ID DBs and read-modify-writes were sometimes getting reset on reboot for existing drives. .844 * [Issue #27363] Fixed null ID processing in the chunk ID helper. .843 (TEST-DC) * Fixed some Box errors introduced in .835. .842 (TEST-F) * For Box, not all 403 errors should trigger a reauthorize. For some of them, show a more meaningful error. .839 * Fixed an upload failure as a result of throttling. * Reenumerate all chunks and recheck for duplicates on existing drives. .836 * [Issue #27362] Fixed an error in duplicate chunk recovery (introduced in .835). .835 * For Amazon Cloud Drive and Box, intelligently handle HTTP conflicts in order to recover from post-upload failures. * For Google Drive, pre-generate upload IDs for new file uploads in order for post-upload failures to not result in duplicate chunks. .834 (TEST-DC) (TEST-PF) (TEST-R) (TEST-V) * [D] When a cloud part would quickly be removed and re-added, the UI would not update properly. .833 * Deprecated the MediaFire provider. It has some major issues right now, including not being able to delete files. * Got rid of deprecated advanced settings having to do with chunk IDs. * Added the ability to recover from duplicate chunks to the chunk ID I/O provider, and enabled for Google Drive. - Due to deficiencies in some provider APIs, some files can go missing when enumerating folders. This can lead to a new file being uploaded with the same name, and if the older file "reappears" afterwards, this can cause issues. * Querying for a chunk's ID is no longer required, and not supported. We always know which chunks should be at the provider, and we will detect missing and duplicate chunks. * There is no "Authoritative" storage mode anymore for the chunk ID storage engines. All storage engines operate identically. - For existing drives, a reenumeration of chunks will take place. * Encapsulated the chunk ID code into a new chunk ID I/O provider, so that all providers that require chunk ID management can easily utilize the same features. .829 * Drives containing duplicate chunks will not mount unless "Allow duplicate chunks" is checked under the Troubleshooting menu. * Optimized database access when bulk inserting many chunk IDs. .827 * [Issue #27339] When a volume is dirty, display a more informative error message and user feedback item. .826 * [Issue #27342] Fixed "An element with the same key already exists" when multiple chunks exist with the same index. .825 * [Issue #27331] Rewrote volume name query code to not use VDS. .824 * [Issue #27344] When multiple files are found for the same chunk, note a warning in the log with the chunk number. .823 * Fixed thread aborts. * Fixed Technical Details window not updating. .822 * [Issue #27342] Updated the Google Drive provider to use the v3 API. * [Issue #27342] Enabled logging of chunk ID enumeration after database reset by default. * [Issue #27342] Added "Allow missing chunks" to the Troubleshooting menu. * Fixed uploads getting stuck. .819 * If multiple chunks exist with the same name, use the first one. .818 * Do not timeout write requests if they are performing local I/O. * [D] Cloud drives were sometimes not shutting down properly, unnecessarily forcing unsafe recovery. * Write timeouts are now measured from the last I/O activity time, not from when the request was started. * [D] Do not allow aggressive cache trimming for the first minute after starting. * When creating new drives or attaching existing ones, volumes named "Cache" or "CloudDrive Cache" will be automatically selected for use as the cache volume. * Use dynamically sized buffers when performing I/O using the File Share / Local Disk providers. * Force Amazon Cloud Drive into authoritative chunk ID storage mode. * The low disk space / throttling user feedback message is now automatically dismissed. .810 * When we know how much space is available for use at the provider, display that in a tooltip when hovering over the "Cloud" or "Cloud Unused" pie slices. * When we know the actual disk space that is used by all of the chunks stored at the provider, display that in the UI. We know the actual disk space used when the provider supports querying for the disk space used by a folder, or when a persistent chunk ID database is used in authoritative mode. * [Issue #27334] For Google Drive and Box, an authoritative local storage engine is required for chunk IDs. - Drives utilizing the legacy RAM-based storage engine will refuse to mount (ChunkId_Persistent = false). - All existing non-authoritative databases will be converted to authoritative databases by indexing all of the chunks stored at the provider. This is a one time process and may take a few minutes, depending on the size of the drive. - For Google Drive, we cannot rely on Google's ability to search for a file because it has been proven to be incorrect at times. * Fixed a security error on reauthorize when the old secret could not be retrieved. * [D] Do not timeout writes prematurely while throttling writes. * [D] The cache data structures are now paged, which dramatically reduces the amount of physical memory that is required per drive in the kernel. * [D] Fixed some memory leaks related to driver loading / unloading. 1.0.0.802 --------- * Added an option to reset databases to the reset settings dialog. .800 * [D] [Issue #27311] Fixed a memory leak in the prefetcher. .799 * Retry when encountering ERROR_NOT_SUPPORTED while impersonating for the File Share provider. .798 (TEST-F) * [D] Generate unique device instance names / symbolic link names in the kernel. .797 * Destroying / detaching a drive shortly after a resize would possibly leave handles open to the cache files. * Fixed a race condition in the drive destruction task that would sometimes leave handles open to the cache files. .796 * [D] Wait for the CloudPart to stop on unmount. * [D] Perform a drive unmount through the bus, not the disk. When detaching through the disk, we can't wait for the CloudPart to stop. .793 * Pinning metadata is now a background child task. That means that it can now be interrupted at any time by other user initiated actions, such as drive resizing. * By default, hide the credential on the credential backup screen, if it was hidden when entered. .792 * Fixed force closing handles on detach. * Fixed a bug in the drive selection bar that would cause the arrows to stop working under some specific conditions. * Fixed some sizing quirks with the first drive created window. .790 * Resynchronized version numbers. .789 (TEST-R) (TEST-V) (TEST-DC) (TEST-PF) * Added a welcome spawner window that shows up when you create your first cloud drive. .787 * [Issue #27289] Added additional confirmation dialogs to drive destruction to prevent accidental activation. .786 * [Issue #27297] When enumerating items, skip explicitly trashed folders and files in Google Drive. * [Issue #27297] Never recreate missing cloud drive data folders in Google Drive. .785 * [Issue #27294] A cache part that was not a multiple of the sector size of the cache drive that it was on would have generated a "cannot align stream" error when performing I/O. * [Issue #27294] Fixed null reference exception in the I/O manager retry control tracing code. .784 * Detaching a drive that still has data left to upload is now allowed. - If the drive has > 10 MB to upload, an additional confirmation dialog is displayed to the user upon detach. - Detaching will begin automatically after all data finishes uploading. - The user can abort the detach at any time from the status bar. * Added task status tool tips to the status bar. * Most tasks can now be aborted from the status bar. .783 * Added driver verification and repair on Windows 10. This will require an extra reboot if the driver needs to be repaired. * Updated Windows 10 detection in MSI. * Updated start menu shortcut. .782 * [D] Retry write I/O on STATUS_PURGE_FAILED until timeout. This avoids write errors on flaky disk controllers at boot time. .781 * Apply CloudFsDisk_Tracing on service start. .780 * [Issue #27272] Chunk ID queries were not optimal. .779 * [Issue #27270] Added a confirmation dialog to the clear cache action. 1.0.0.778 --------- * Fixed the NT API calling convention. This was breaking the x86 build. 1.0.0.777 --------- * [D] [Issue #27251] Fixed a race condition in the write I/O path that was causing a system crash when writes to the cache disk were failing and being retried. .776 (TEST-F) * [D] Fixed a deadlock when destroying a drive. This would sometimes leave the CloudPart... folder in use after destruction. .775 * Fixed encryption tooltip when automatic unlock is disabled. .774 * If a Store object ID changed while an asynchronous task was preparing to run, the task would fail. - One practical example of this was: If the VDS disk information changed while reauthorization was taking place, reauthorization would fail. .773 * Persist the connection name after reauthorizing a drive. .772 * Fixed detach delay. .771 * Fixed attaching legacy Amazon S3 drives. .770 * Added additional API keys for Google Drive. Reauthorize existing drives to use the new API keys. .768 * Fixed legacy Amazon Cloud Drive authorizations. .767 * Fixed cache alignment descriptor retrieval. .766 (TEST-R) (TEST-V) (TEST-DC) (TEST-PF) * [Issue #27190] Added the ability to specify a custom path for Box in ProviderSettings.json. * [Issue #27190] Added ProviderSettings.json to configure various provider settings. Located in %PROGRAMDATA%\StableBit CloudDrive. .763 * [Issue #27234] Always align unbuffered I/O to physical sector size boundaries for all drives. This was causing "The parameter is incorrect" error to show up under certain drive configurations. On most drives, unbuffered I/O was already being properly aligned, but when performing direct provider to cache I/O (optimized path), then it was not necessarily being aligned. * Added ValidateLogins_Timeout .config setting to control how long the service will wait for the provider to respond when refreshing credentials. * Refresh providers / credentials in parallel for faster initial connection and faster credential refreshes when many cloud drives are involved. .760 * [Issue #27233] On a fresh install or an upgrade, do not wait for Internet on service start. .759 * [Issue #27224] Fixed uploads getting stuck in WholeChunkIoImplementation. * [Issue #27224] Dynamically scale the upload / download buffer size depending on the transfer speed. This prevents transfer timeouts on slower connections and lowers the transfer overhead for faster connections. .758 * Fixed multiple issues with attaching encrypted drives. * Allow weak encryption passphrases after showing a warning and user confirmation. .757 * [Issue #27173] Added an explicit warning to make it absolutely clear that you will need the encryption key to decrypt your data. .756 * Fixed 404 error when attaching legacy drives from Microsoft Azure Storage. .755 * [D] Added deadlock mitigation code on write. It's used for working around buggy filter drivers that cause deadlocks for virtual disks that store their data on another volume. .748 * Tweaked the UI for the Essentials Dashboard. .745 * [Issue #27222] Fixed memory leak in file ID storage. .744 * When the upload threshold is not met, hovering over the "To upload" item in the UI will indicate that we're waiting for more data to be written to the drive. * All cloud providers (including all existing cloud drives) will now set the upload threshold to 1 MB by default, with a 5 minute timeout. * Added upload threshold minutes setting to the I/O Performance window. - Normally, when upload threshold is enabled, uploading will not begin until the amount of data that needs to be uploaded exceeds the set upload threshold. - When upload threshold minutes is set to > 0, if the amount of time that elapsed from the first write is greater than the upload threshold minutes, uploading will begin regardless of whether the upload threshold has been met or not. This is designed to periodically flush the "to upload" queue in order to give the cloud drive greater resiliency if the local disk fails, even when the upload threshold is enabled. .742 (TEST-PF) (TEST-R) * Sometimes Google Drive will incorrectly flag a chunk as malware or spam and will prevent it from being downloaded unless "acknowledgeAbuse" is set to true. * Added a persistent file ID storage database for providers that don't have the ability to download files by name (Box, Google Drive): - This is designed to reduce the number of API calls that need to be made to these providers when downloading chunks. - Each cloud drive now has its own local persistent file ID database. - The new persistent database can operate in 2 modes, authoritative and non-authoritative. - In authoritative mode, the provider is never queried for file IDs because the database knows about all existing IDs. - In non-authoritative mode, chunk IDs will never be re-queried again once they are looked up once. This is less efficient than authoritative mode but allows for backwards compatibility with existing drives. - All newly created drives will operate in authoritative mode. - All existing drives will operate in non-authoritative mode. - If the file ID database is damaged or missing, the drive will be converted to use non-authoritative mode. Losing the file ID database will have no impact on the data integrity of the cloud drive. - When upload verification is enabled, chunk IDs will no longer be re-queried from the provider when verifying a chunk. This behavior can be changed by setting ChunkId_RequeryOnVerify to True. - Persistent file ID storage can be completely turned off by setting ChunkId_Persistent to False. This will revert back to the legacy in-memory file ID cache. Re-enabling the persistent store after disabling it will put all cloud drives into non-authoritative mode. * Disabled upload verification by default for most providers. .726 * Fixed inactive border color on Windows 10 / Windows Server 2016. .725 * Fixed throttling icon. .724 * Treat "rateLimitExceeded" 403 error from Google Drive as throttling. .722 * For read-modify-write, do not validate checksum on read. Since we'll be overwriting the data anyway, it doesn't matter. .721 * [Issue #27170] When saving the encryption key backup, explicitly flush the volume handle when dealing with local disks. .720 * If pinning settings change while the pinning task is running, restart pinning and always use the latest queued state. .719 (TEST-R) (TEST-PF) (TEST-V) (TEST-DC) * Added various tooltips. * Re-enabled the Amazon Cloud Drive provider. Since the API keys are now on an invite only basis, this provider is now meant to be used by Amazon for their application approval process. .718 * Fixed user name or password is incorrect error for the file share provider. * [D] Added additional tracing for IOCTL_SCSI_MINIPORT. * [D] Added support for IOCTL_SCSI_GET_ADDRESS. .714 * [D] [Issue #17737] When multiple drives need to perform unsafe recovery, perform recovery one drive at a time, but for the queued drives, display a message in the UI indicating that a drive is queued for recovery. * [D] [Issue #17737] Unsafe recovery mounting should happen on a separate work queue so as not to block safe mounting. .708 * [Issue #27153] Do not allow new cloud drives to store their cache on removable disks. .707 * Improved the experience of selecting a FTP path when the root directory is not the home directory. Browsing a FTP path will now default to the home directory unless "/" is explicitly entered into the path text box. * Added a maximum connections setting to the I/O Performance window for providers that support connection limiting (FTP only for now). - Defaults to 2 for FTP (Ftp_DefaultMaximumConnections). - FTP uses connection pooling in order to reuse existing connections for future request. But without this enabled, new connections will be spawned as they are necessary, and this may cause excessive load on the FTP server. This setting limits the total number of connections that can be spawned in the connection pool. - The maximum connection setting is per drive, and not any sort of combined limit. - Other non-drive related tasks, such as refreshing the list of available drives in the New Drive tab, will open up their own connections outside of this limit. Those connections are not pooled nor limited and they will be closed as soon as they are no longer needed. * Fixed some issues with FTP. .699 * [D] [Issue #27157] The system would crash if a partially mounted disk was present while the system was shutting down. .698 * [Issue #26008] Fixed issue with custom paths in the local disk provider. .697 * Added FTP provider: - Supports FTP / FTP + SSL / SSH + FTP. - Supports client certificates (PKCS #12) for FTPS and SFTP. - Supports client private key (PKCS #8 / OpenSSH / PuTTY) for SFTP. - Supports Kerberos / NTLM authentication for SFTP. .696 * [Issue #25999] Fixed the ability to install into a non-default folder (INSTALLLOCATION property passed to MSI now overrides the default). .694 * [Issue #26008] Added the ability to specify custom paths for the local disk provider. .693 * [D] [Issue #26048] Added an advanced option that allows the cache to be stored on the pool (CreateDrive_AllowCacheOnDrivePool). .692 * [Issue #27055] Updated the WiX Toolset to the latest stable version (3.10.3). .691 * In the remote control drop down, mark the local computer explicitly, and make sure that it's always available as an option. .690 * [Issue #25986] Fixed numerous issues with remote control: - The list of computers used to lose systems periodically, leading to one way visibility between two systems. This was due to DNS lag. - Reduced chatter between systems. - Adjusted keep-alive retries for busy networks. - Remote control cannot be turned off when connected to a remote system. * [Issue #25986] Removed some extraneous code that was running on UI window exit. .689 * [Issue #27125] Change the default save file type for the encryption key backup document to .PDF. .688 * [Issue #27105] Avoid runaway thread issue in the performance sampler when there is a Store deadlock. .687 * [D] Notify the service when low on disk space on the cache drive (< 5 GB). If uploading is paused, it will resume. * Detaching a drive can now be aborted (if waiting for uploading to complete). * Added "write threshold" option to I/O performance. .685 * Attaching a drive (non-force attach) will now perform an extra check at attach time to make sure that the drive is not attached to any other system. .684 * Tweaked the bandwidth throttling algorithm to allow for "burstiness", which makes it more accurate under fluctuating bandwidth conditions (IoManager_ThrottlingCatchupS). The set limit can now be temporarily exceeded in order to "catch up" due to a previous slow down. Set to 0.0 to disable. * Actions in user feedback can now show up as disabled depending on the application state. Each action defines its own rules of availability. .682 (TEST-V) * Fixed issue with internal validation due to missing FTP library. .680 (TEST-DC) (TEST-R) (TEST-PF) * Switched the default cluster size for new NTFS drives to the Windows default. The 64 KB cluster size generates a lot of fragmented writes, which are suboptimal for StableBit CloudDrive to process (because of CPU, memory and provider traffic overhead). * Profiled for CPU usage and made multiple changes based on the results: - Simplified and re-factored write pre-buffering and write thread boosting once again for faster I/O reads from the cache (when uploading). - Deprecated the managed CRC32 checksum algorithm for new drives. SHA-1 replaces it where data authenticity is not required, using the native Microsoft CSP. - Upload verification now uses in-stream hashing, when pre-buffering is used, in order to speed up verification. - Increased various internal memory buffers used for I/O to 1 MB (from 128 KB). This lowers CPU usage and increases throughput. - The intention here is to optimize StableBit CloudDrive for Gigabit speeds and beyond when running on a modern system using a SSD as the cache drive. * Removed the ability to create new Amazon Cloud Drives, as per the new Amazon policy change to invite only API keys. * Rewrote write request processing code to take advantage of Ranges in order to divide up the incoming write requests along chunk sized boundaries (if applicable). * Rewrote write thread boosting to use a separate thread pool for boosted write threads. .667 (TEST-DC) - Now with Amazon Cloud Drive and error simulation as part of the test. * Added various error simulation options to Technical Details. * Added read-ahead pre-buffering when reading from the on-disk cache to write to the provider. - Data for the next provider write request will be pre-buffered while the current write request is being processed. - There is now a single dedicated pre-buffering thread for all write requests, per drive. - Pre-buffering respects the I/O priority setting of the upload threads. * [D] Added a non-paged caching layer for read requests that can be serviced 100% from the provider. - When a read request needs to be serviced by the provider, previously, the data would be written to the on-disk cache by the system service and then subsequently that data would be re-read back in by the driver to complete the read request. - With this optimization, the data gets written to the cache, and at the same time, in parallel, gets written to a non-paged memory buffer that will be used to complete the incoming read request. - This optimizes the overall drive read speed for slower cache drives. * Automatically clean up unused credentials. * Fixed task errors that were showing up due to having multiple connections to the same provider using the same credentials. * Added chunk read short circuiting, eliminating re-downloading of the same chunk range from different threads. .642 * Fixed deadlock in cloud drive mounting. * Added JSON serializer/deserializer to the Store for faster service startup times. .636 * Fixed incorrect I/O context being used for I/O request tracking in the Technical Details window. .635 * [Issue #27088] When the bytes per cluster did not equal the bytes per sector, NTFS metadata pinning would not work. .634 * You must now obtain your own API keys to use Amazon Cloud Drive. Bandwidth throttling is now disabled when using your own API keys, and the thread limits have been raised to 20. Existing drives will continue to work as they did previously (with the old limits in place). .631 * ReFS is not supported on x86. .630 * Added the ability to create ReFS cloud drives. - ReFS drives can be created starting with Windows 8 and newer (both in client and server editions). - ReFS drives cannot be attached to OSes older than Windows 8. - ReFS volumes can be up to 1 PB in size (and are not limited to 256 TB like NTFS). The 1 PB limit is the maximum drive size in StableBit CloudDrive and is not a ReFS limit. - The metadata pinning engine cannot yet pin ReFS directory entries. .629 (TEST-V) * Fixed an internal validation error (due to the addition of read thread boosting). .628 (TEST-DC) (TEST-PF) (TEST-R) * [D] Unregister from volume change notifications prior to performing pre-shutdown / shutdown. .627 * [D] [Issue #27079] Added a new option (CloudFsDisk_IsQuickRemoval) that controls whether removable cloud drives default to the "Quick removal" policy by default. This does not apply to drives using the Local Disk provider, those drives always default to "Better performance". The Removal policy can be changed at any time, on a per-drive basis, by opening Device Manager and going into the cloud drive's disk properties. This is True by default. * [D] [Issue #27079] Added a new option (CloudFsDisk_IsDriveRemovable) that controls whether cloud drives advertise themselves as removable drives to PnP. Removable drives expose the "Removable policy" tab in the drive's property window (in Device Manager). This is True by default. .624 * [D] [Issue #27069] Added CloudFsDisk_MaximumConsecutiveIoFailures .config setting which controls how many unrecoverable I/O failures will cause a drive to forcefully unmount. The default is 3, and you can set it to 0 to disable forceful unmounting (use with caution as this can lead to system lockups). .623 * [Issue #27077] Amazon Cloud Drive returns a HTTP 400 error (bad request) when a chunk appears to get lost on their end. The error code is returned as a JSON encoded string inside a JSON encoded object as part of the 400 error. We now parse these, and process them accordingly. The user will now get a more informative "Chunk N was not found" error in the UI. The data loss can be verified by logging into the Amazon Cloud Drive's web UI and attempting to download the lost chunk. There is no mitigation for this issue, other than displaying a more informative error message, because the data is lost on Amazon's end. .621 * Fixed range error when mounting. .620 * [Issue #27053] Default to 4096 bytes per sector for new drives. * [Issue #27053] Do not allow the local cache to be stored on a disk that has > bytes per sector than the cloud drive being created. * [D] [Issue #26046] To minimize RAM usage at the cost of cache resolution, decreased the number of blocks that the local cache will use (in memory) for drives < 8 TB. This will reset cache usage data. .618 * Added a cluster size option to the create a new drive window (defaults to 64 KB). .616 * Provider I/O errors encountered while mounting will no longer be retried to speed up the overall mounting process. .615 * Lowered the amount of time to wait for an Internet connection on boot to 30s (CloudDrive_WaitForInternetS). * Re-enabled partial non-exclusive writes for the local disk provider. * Fixed excessive memory usage by the service (introduced in .609). * [D] When a force unmount occurs, do not immediately remount the drive, even if all of the pre-mount checks pass. Instead, give the user a chance to approve the mount manually. .610 * [D] Fixed some miscellaneous issues in the driver having to do with drive unmounting. * [D] If Plug and Play will not finish force unmounting our drive after 30 seconds, force an internal safe unmount of the cloud part and fail all future I/O requests from Windows. This can happen if something has a direct handle open to the disk. * [D] [Issue #27053] When 3 consecutive unrecoverable I/O errors occur in the kernel driver, the drive will be force unmounted from the system. For example, if a read request can't be serviced because the data in the cloud is not accessible, we have to fail that read request, and that's considered an unrecoverable I/O error. Disk read timeouts due to a non-responsive system service are also considered unrecoverable I/O errors. The force unmount is managed entirely by the kernel driver and does not involve any user mode component. The force unmount occurs in order to prevent the system from locking up in cases where the drive's data becomes inaccessible (for various reasons) after the drive is successfully mounted. A force unmount will not trigger an unsafe start on the next mount. The drive is actually safely unmounted from the system, and the only part that is forced is Windows Plug and Play. * [Issue #27070] Fixed drive resizing not working. .609 * [Issue #27053] Error reports will now be written out when a drive mount fails due to a problem with the provider. * [Issue #27053] When a drive mounts, it will now perform a check to make sure that it can properly communicate with its data provider. If it can't, due to a security error or some other type of error, it will go into a pseudo-mounted state. In this state, the data structures necessary to work with the drive will be loaded, but no virtual disk will be spawned in Windows. - The user will be informed via the StableBit CloudDrive UI that there was a problem mounting the drive, and will be offered different options to resolve the issue, depending on the nature of the error. - If there is a security error, the user will be offered the option to reauthorize. If the reauthorization succeeds the drive will be mounted. - If there is a communications error with the provider, the user will be offered the chance to correct the problem and to retry the mount. - At all times, there will be an option available to destroy the drive from the local system entirely. If there is a communications error with the provider, the drive's data that's stored at the provider will not be deleted. That data can be deleted at a later time, once communications is restored. .608 * [D] [Issue #26051] Fixed formatting issue. * [Issue #27053] Error reports will now be written out on a final read I/O failure. * Default to 512 bytes per sector for new drives. .605 * [D] [Issue #26051] Added support for SCSIOP_VERIFY16 to the virtual disk driver (issued by IOCTL_DISK_VERIFY). * [D] Added function-level tracing to the driver. * Do not update the free disk space while the service is shutting down. * [D] Immediately after resizing a drive, the cache's blocks were not being aligned to the file system compression units. Rebooting would fix the issue. .599 * [D] [Issue #26045] Removed usage of partial MDLs on write. .598 * [D] The kernel prefetcher is now able to perform special aligned (long range) prefetch requests. These can be as large as 100 MB per request, and multiple requests are permitted. These special long range requests are only allowed if the entire prefetch range can perform prefetching. If the long range prefetch check fails, then a standard more granular prefetch takes place (using 512 KB blocks). The more granular prefetch is now optimized to combine multiple smaller requests into one or more larger requests, if possible, in order to reduce overhead. - Aligned long range requests are possible under these circumstances: - If legacy chunk verification is used, alignment is set to the chunk size. - If block based chunk verification is used, alignment is set to 1 MB. - If minimum read size is specified, then alignment is set to the minimum read size or to the above, whichever is greater (use this with caution as it can reduce the effectiveness of multi-threading while downloading, depending on your settings). .597 * When chunk caching is enabled, avoid downloading the same range of data multiple times using the new Range locks (especially useful when minimum read size is enabled). * Added a value lock that can perform thread synchronization based on a Range's value, utilizing the new Skip List. * Wrote a comprehensive Ranges implementation on top of the new Skip List to efficiently perform set based operations on a large number of ranges. This implementation exposes basic set theory operations, but instead of performing them on discrete values, it performs them on ranges of values. This is much more efficient and most set operations are performed in O(log N) time. * Wrote a new Skip List data structure that has the desirable performance qualities of a binary tree, but doesn't require balancing. Adding / removing items is O(log N), and all items remain sorted. Items can be enumerated backwards and forwards like a doubly linked list. * Added read thread boosting which is invoked when we know for sure that the current read thread will not end up issuing provider I/O and is simply waiting for another thread to finish processing. * [D] Tweaked write throttling speeds. .592 * Added the new cache settings to the Drive Details window. * The "Off" setting for the local cache is now called "Minimal". .591 * [D] Added 3 cache types: - Expandable: Limited read cache, unlimited write cache (same as before). Best for performance. - Fixed: Limited read / write cache, where writes overwrite the read cache. - Proportional: Limited read / write cache, but writes are conceptually separate from reads. Reads can still utilize the write portion of the cache, when not in use. .584 * [Issue #25958] For Amazon S3, creating a new drive would not work for certain regions. Moreover, due to a bug in the AWS SDK, drives that failed creation in this way would get stuck in a "Destroyed" state with no way to delete them. .583 * [D] Replaced the existing read-modify-write recovery implementation with a new one. The new implementation coordinates with the kernel driver which now extends the existing I/O architecture to support the notion of temporary writes to the cache. The system service is no longer solely responsible for managing read-modify-write failures by itself. Instead, it now coordinates with the driver to ensure that no data loss occurs for read-modify-writes. Even in the face of total service failure (or an entire system crash), consistency is always maintained for data on the drive. - [D] Temporary writes have a brand new module in the driver that coordinates I/O among all of the other modules (reads, writes, trims, resizes, etc...) - [D] Temporary writes will be promoted to full writes if a read-modify-write explicitly fails, if the service crashes or if the system crashes. - [D] Temporary writes never overwrite existing newer data in the cache. Instead, they fill in "holes" in the cache to prevent provider data from getting lost due to a potential future I/O error. - [D] Temporary writes are added to the local cache as low priority data and will get trimmed first, once it's safe to do so, and the cache becomes full. - Advantages over the old recovery system: - The new system is anticipatory, not reactionary. In that, it anticipates a failure and sets up for recovery prior to the failure occurring, instead of only performing recovery processing once a failure has occurred. This means that even if the entire system crashes, recovery still takes place correctly. - The new system is more efficient. It only stores recovery data from the cloud, for areas of the disk that we don't already have cached locally. - The new system is tightly integrated with the existing I/O model in the kernel and reuses the existing cache instead of relying on external data. * [Issue #25998] Added tooltip to the drives drop down on the Technical Details window. .579 * [D] The pre-shutdown sequence now coordinates with the main system service in order to perform pre-pre-shutdown processing. All write threads are aborted before pre-shutdown begins. This is safe to do because writing to the provider is not required during either per-shutdown or shutdown. It is actually desirable not to have write threads active while the main shutdown sequence takes place because main shutdown only has a finite amount of time to finish processing before the OS aborts it. * When performing a multiple-I/O aligned write, consider a read-modify-write to be eligible for recovery, even though technically it is just a write. That's because it's part of a larger sequence of events that involve read-modify-writes that need special recovery processing. .564 * [D] Write IRPs should be marked as paging I/O. .560 * [D] Read IRPs should be marked as paging I/O. .559 * [Issue #25995] Allow Google Drive and Google Cloud Storage to use random open port seeking when performing local OAuth. Dropbox still requires standard ports (8080, 8081, 8090) for local authentication, but will fallback to server authentication if those ports are not available. .558 * Disable buffered access to the local cache files. .557 * [D] Perform unbuffered I/O when reading from the local cache. .556 * All new Box drives will force chunk verification to be enabled. .555 * In Box, chunks can go "missing" after an upload for an indeterminate amount of time, or perhaps indefinitely. If an upload is reattempted again when a chunk goes "missing", the results are indeterminate. Either the next upload will go through or it will fail with a conflict. The conflict may not necessarily reference the file that was just uploaded previously, but an older file that has also gone "missing". If it does go through, then we end up with 2 or more files with the same name. In this case, downloading the file later is also indeterminate, because we can't know for sure which file we will get. - After an upload, re-query the chunk ID to make sure that it matches the one that was just uploaded. - If not, repeat indefinitely until the write times out and fails or the correct ID is returned. - A Warning will be logged: "Chunk has gone missing. Re-querying chunk ID..." .552 * Added option to Technical Details, to stop uploading on write verification error. .550 * Hash and verify persisted failed writes on load. .549 * Additional housekeeping code for failed writes. .548 * On service exit, cleanly abort all write threads and perform any final processing that's necessary for each implementation. * After uploading a file to Box, sometimes, very rarely, Box issues a 409 conflict even when we're absolutely sure that the file that we're uploading has a unique name that has never been uploaded before. Unlike a normal 409 on Box, this one doesn't contain the ID of the file that is in conflict. Reissuing the same upload then returns the ID of the file in conflict, but oddly enough, this ID has never been seen before, and seems to belong to the new file that was being uploaded. So it seems that the file was in conflict with itself. In these cases, because we're not exactly sure what state the file is in, delete everything and retry the upload. This has been reproduced in a sandbox. * When deleting files on Box, delete them permanently and not to the Trash (2 API calls). .547 * [D] Fixed a race condition that was causing a KERNEL_APC_PENDING_DURING_EXIT crash on reboot. .545 * When handle caching is disabled, explicitly close all handles to the local cache when a drive unmounts. Do not rely on the garbage collector. * Persist ReadModifyWriteRecoveryImplementation data to disk. - When a "Critical I/O" (read-modify-write) failure occurs on system shut down, there is a chance that some of the recovery data will still be in memory when the service safely shuts down. Previously, this data was not persisted to disk and on the next service start, the same read-modify-write will be retried and will not participate in the recovery process from the previous failure. - In order to prevent this issue, all read-modify-write recovery data will now be persisted to disk and reloaded on a drive mount. .542 * Fixed a race condition in the thread pool thread boosting code that was causing a service crash. .541 * Fixed a race condition in read->modify->write recovery for the WholeChunkIoImplementation. .539 * [D] [Issue #25979] Fixed a system crash on drive resize. * Fixed short circuiting in the block based checksumming / cryptographic signatures implementation. .537 * Added a minimum download size option to the Create and Attach drive windows, as requested: - When set, all data downloaded from a provider (excluding upload verification) will be at least of this size. - Data that is not needed for the current read request will be cached in RAM. - The minimum download size cannot exceed the chunk size. - Drives employing chunk validation already have an implicit minimum download size of 1 MB. - Legacy chunk validation has an implicit minimum download size of the entire chunk size. - To apply the minimum download size to an existing drive, you will need to detach and re-attach the drive. The limit can be set in the Attach drive window. .536 * Fixed crash on service start. * Never allow null chunks to be uploaded to providers that are encrypting their data and that do not support partial writes. * Do not delete chunks in Box that are null chunks. .534 * Keep alive checker was generating excessive warnings on ping failure. * Lower the number of ports to seek for openness when performing keep alive from 100 to 10. * On drive resize, do not run the resize code if the only thing that needs to be updated is the cache size. * Reauthorizing over remote control was showing an error message, even though it was working properly. .533 (TEST-DC) (TEST-V) (TEST-PF) * Internal validation now uses separate logging and tracing settings. The validation log file will be allowed to grow up to 10 GB and will output more useful chunk I/O tracing data. * Added some additional permutations to internal validation that don't require I/O alignment. * Increased internal validation I/O count to 10000 per permutation. * Simulate read / write errors when performing internal validation (3% chance of a read / write error). - Some data inconsistencies only show up when faced with less than perfect provider I/O. - This test now validates data consistency in the continued presence of provider errors. * Added ReadModifyWriteRecoveryImplementation with the ability to properly recover from read->modify->write failures. - When a read->modify->write fails on the "write" step, the data on the provider becomes indeterminate in that entire write range. - Because the read->modify->write expands the originally requested write range, that inconsistency is now outside of the range that's being written to by the I/O manager. - On the next retry, the "read" step will read in this inconsistent data and it won't have any way of recovering the correct data, because that data is outside of the current write request. - The new ReadModifyWriteRecoveryImplementation fixes this issue across all implementations that perform read->modify->writes. .518 * Added a write thread boosting optimization to all cloud providers. If different parts of the same chunk need to be written to a cloud provider, temporarily boost the number of write threads in order to combine all of those writes into one. This is not applicable to the Local Disk and File Share providers. * Instead of waiting for them to finish, abort existing uploads on service exit. * Added upload / download throttling settings to the I/O Performance window. .517 (TEST-V) (TEST-PF) * Disabled handle caching for the Local Disk and File Share providers. * Power failure test now tests I/O under normal shutdown conditions as well as abnormal power loss. * Added the ability to turn off certain provider I/O optimizations in the Technical Details window. .513 * Do not crash the service when there is no secret for the Local Disk provider. * Remove non-displayed user feedback messages from the persistent store. * Moved write control functionality from the Troubleshooting menu to the Technical Details window. * Added a new "Service Log" window that's accessible from the Technical Details window. .511 * Reduced unnecessarily excessive service memory usage (disabled pre-buffering) for uploads to Google Drive and OneDrive (v2). * [D] Disabled local cache trimming range expansion. This should improve local cache performance. * [D] Drives larger than 3.125 TB were not aligning their local cache to the NTFS compression unit size. All local cache units will now be aligned to the volume compression unit size (the exact value depends on the cluster size of each volume). As a result of this change, the cache map will be automatically rebuilt on all cloud drives using the new formula to compute the correct cache unit size. Cache usage history will be reset. .509 * [D] Trimming the local cache should yield to I/O if it takes too long. .508 * Parallelized drive deletion for Local Disk and File Share providers. * Removed padlock icon for obfuscated drives because it may induce a false sense of security. To secure your data, enable full drive encryption. * Added an additional log warning when the HTTP mime type error workaround is attempted on Google Drive. * Added additional log warnings when HTTP error codes are returned. .506 * Added drive deletion cancellation ability to the Local Disk and File Share providers. .505 (TEST-DC) (TEST-V) (TEST-PF) * All new unencrypted cloud drives will be encrypted with AES-128 CBC using a static public key. This isn't done for security, but to give the stored data a random appearance (e.g. to prevent providers from guessing file types, which can lead to issues). .504 * Added the IoManager_DefaultMaximumReadAggregation setting to control the maximum read I/O aggregation for providers that don't explicitly specify one. * Increased maximum read I/O aggregation for the File Share and Local Disk providers to 100 MB. .503 * When many I/O requests completed rapidly, excessive memory and CPU usage was observed due to request / thread tracking. Added ChunkIoPipelineTrackingImplementation_MaximumInFlightIoToTrack, ChunkIoPipelineTrackingImplementation_MaximumClosedIoToTrack, Performance_MaximumClosedIoToTrack advanced settings to limit request tracking. * For the Local Disk and File Share providers, the periodic releasing of handles was sometimes causing a deadlock. .500 (TEST-V) * Added more comprehensive validation testing to service: - To start: "CloudDrive.Service.exe /validate" - Validates various chunk I/O pipeline permutations using simulated I/O. - Validates cloudfs_disk.sys driver by running all internal tests. * Added explicit buffer management requirements that all chunk I/O implementations must abide by. Any violations will abort the I/O request with an error. .497 * Reimplemented short circuiting as a chunk I/O pipeline helper into the various implementations that lock on chunk writes. .494 (TEST-DC) * [D] Fixed a rare but critical race condition that was causing provider I/O reads to be written incorrectly to the local cache. This was not affecting writes to the provider, just reads from the provider. * Be more persistent when deleting cloud parts after a drive detach / destroy. .486 * Reduce flicker of cloud drives drop down in Technical Details. * Reduced the default chunk size for MediaFire to 1 MB. .485 * Rewrote stream to stream copying. .484 * Re-authorization was not working for MediaFire. .483 * MediaFire sometimes returns negative numbers for how much storage space is currently in use. * Technical Details windows pause button was not working correctly. .480 * Added a drive details dialog to the technical details window. * I/O Threads is now known as "Technical Details" and is now accessible outside of the troubleshooting menu. .479 * Fixed a bad interaction between the unit checksum implementation and the aligned chunk I/O implementation causing buffer position misalignment under some circumstances. * Fixed unit cache chunk I/O implementation losing buffer position. * Fixed aligned chunk I/O implementation, when dealing with requests smaller than a block. * Added a "short circuit" chunk I/O implementation. - If while a chunk is being written to an I/O provider, the same chunk needs to be read, then temporarily pause the upload, read in the parts that we need, and resume. * Added tracking of I/O requests, in addition to provider I/O, to the I/O Threads window. - Depending on the Chunk I/O pipeline configuration of the particular provider in use, I/O requests can spawn one or more other child I/O requests. - Some I/O requests can be serviced instantly (e.g. cache), while others end up being serviced by the provider. - Each I/O request can be in a particular state. Each request now shows the state that it's in, and a tool tip explains exactly why it's in that state. - I/O requests that were assisted by the chunk cache will be shown in blue. - I/O requests that were short circuited will be shown in magenta. - The display can be paused for further examination. .470 * Fixed excessive OAuth 2.0 token refreshes. * Fixed secret splitting on service restart. * Updated MediaFire token handling. .469 * [Issue #25919] When the cloud part folder is missing from Google Drive, do not crash the service on startup. .468 * Updated the I/O Threads window. .467 * [Issue #25923] When overwriting an existing chunk on Google Drive, sometimes the server returns this undocumented error: HTTP 403 - "The requested mime type change is forbidden." If this happens, delete the underlying chunk and reattempt the upload as a new file. .466 * Enabled experimental MediaFire provider. 1.0.0.463 --------- * Fixed access denied when destroying a cloud drive that was using the File Share provider. .462 * Use separate HTTP timeouts for reading / writing (IoManager_HttpConnectionReadTimeoutMS / IoManager_HttpConnectionWriteTimeoutMS). .461 * After unlocking a drive, the UI would be blank until it was resized. .460 * Fixed null reference when server is throttling us outside of the regular read / write I/O path (e.g. deleting a drive). * When a task would throw an Exception that was defined in the service, the UI would not be able to load that Exception information and would get stuck in an unusable state. * Tweaked drive deletion for OneDrive for Business to cancel quicker and to resume deletions where it left off instead of from the beginning. .459 * Fixed some minor data validation issues in the I/O Performance window. * Amazon Cloud Drive was not working properly with the new Chunk ID lookup system. .457 * Chunks were getting written right after a drive's destruction. When using some providers (e.g. Google Cloud Storage), this could have left behind some data even after a drive was destroyed. Refreshing a connection on the New Drive tab would have listed a destroyed drive in a destroyed state. This is now fixed and won't happen any longer. .456 * Added upload / download throttling indicator (little turtle) to show when uploads / downloads are being throttled by the server. .454 * [Issue #24913] Implemented drive name synchronization between the OS volume label and the cloud drive name. .453 * [D] Fixed encrypted drives showing up as "Offline" after mounting and exhibiting I/O errors on access. This was the result of a race condition where the encryption key was getting cleared right after a mount. .452 * Unit chunk cache now defaults to 20 MB for all cloud providers. * Deprecated the old chunk based in-memory cache and replaced it with a more effective unit based cache. - Units can be smaller than a chunk, and can cache multiple regions within the same chunk. - Unit sizes depend on the verification algorithm used: - For block based checksumming or cryptographic signatures, the unit size is the size of the checksum unit (defaults to 1 MB for now). - For the non-block based legacy checksumming algorithm, the unit size is the chunk size. - For non-verified drives, the unit size is 1 MB or the chunk size, whichever is less. - When the cache is enabled, all provider I/O is forced to be at least of unit size. - All existing drives will get the benefit of this new cache, not just new drives created after this build. - Tracing areas: UnitCacheChunkIoImplementation, UnitCache .451 * In order to improve download thread utilization, disabled exclusive chunk reads for: - Amazon Cloud Drive (experimental) - Box - Google Drive - Microsoft Azure Storage - OneDrive (v1 legacy provider) -- - All other providers except Microsoft OneDrive for Business (which won't benefit from this) already had this disabled. - The providers listed above should experience better download performance when using higher download thread counts. .450 * [Issue #24914] Do not allow chunk sizes > 20 MB for any newly created cloud drives using the Google Drive provider. .449 * [Issue #24916] Fixed Google Cloud Storage "Security error". - Google Cloud Storage buckets exist inside of "projects", StableBit CloudDrive could not automatically manage (list or create) the buckets necessary to store your data in Google Cloud Storage without knowing your Project ID. - Added support for Google Cloud Storage "Project IDs", when you want StableBit CloudDrive to manage buckets for you automatically. In this mode, there will be exactly one bucket created for every cloud drive. - Added the ability to use folder-based storage inside of a single bucket of your choosing, without having StableBit CloudDrive managing the buckets for you. This requires you to use the Google Cloud Platform Console and create a new bucket, specifically for StableBit CloudDrive use. In this mode, multiple cloud drives can coexist in a single bucket. * Lower the allowed "To Upload" on detach to 100 KB. .446 * Rewrote the I/O Performance window to use Logic. * [Issue #24912] Removed the 8K sector size option. .444 * [Issue #23907] Prepend a null byte to all Google Drive chunks, for drives created after this build. .443 * When connecting to a provider or reauthorizing fails, display an explicit error message in addition to the errors in user feedback. .442 * Added AmazonCloudDrive_ClientId and AmazonCloudDrive_ClientSecret advanced settings for testing purposes. * For providers that don't support downloading by file name, implemented a centralized file ID translation and caching framework: - The default caching engine is using RAM (maximum amount can be adjusted using ChunkId_MemoryLimit). - Multiple storage engines can be supported, including persistent storage such as a database. - Providers: - Amazon Cloud Drive (experimental) - Box - Google Drive .441 * Turn off exclusive reads for OneDrive to improve multi-threaded read performance. Exclusive reads ensured that no more than one thread was able to read the same chunk. This is no longer required with the new API. .440 * [D] Wait 5 minutes (up from 2 minutes) before giving up detaching a drive. * Encryption related fixes when not saving the unlock key locally. * OneDrive sometimes returns HTTP 403 errors with the headers: - X-ClientErrorCode: VirusError - X-Virus-Infected: ScanError - This happens even though the data being uploaded is encrypted and is clearly not a virus. Apparently these don't go away on retry and we have to deal with them: - All new cloud drives created with the OneDrive provider will prepend a null byte to all chunks. - This change is backwards compatible, but the old format may cause this error. * Added the ability to control tracing levels from the Troubleshooting menu. .436 * Added troubleshooting options to stop uploading on encountering a specific error. .431 * Fixed chunk verification adding superfluous null padding at the end. * Added troubleshooting option to stop uploading on encountering the first error. * Fixed race condition in chunk verification causing spurious verification failures. .423 * Block based chunk verification algorithms can now do both block sized reads and writes. So for providers that support partial writes, it's no longer necessary to perform whole chunk I/O writes if chunk verification is enabled. * MAC chunk verification and AES encryption keys are derived using HKDF-HMAC-SHA512 (RFC-5869) from the master key. * Added support for HMAC-SHA512 chunk verification for encrypted drives. - Checksum verification is now called "chunk verification" and will use different algorithms depending on the scenario. - For unencrypted drives, CRC32-Block will be used to check data in the cloud for corruption. - For encrypted drives, HMAC-SHA512-Block will be used to check data in the cloud for authenticity. - HMAC-SHA512-Block uses the native NT CNG provider. * Added the aligned chunk I/O implementation that can automatically align I/O to unit sized boundaries for either reads / writes or both. - 2 alignment strategies are supported: - Minimize memory usage at the cost of additional I/O calls. - Minimize the number of I/O calls at the cost of additional memory usage. - An automatic strategy can dynamically select the correct strategy to use based on each I/O request. .422 * The new OneDrive V2 provider is no longer experimental. .421 * Fixed "The constructor to deserialize an object of type ... was not found" when a custom provider error was thrown. * For the Local Disk provider, checksumming can now be optionally enabled for chunks up to 100 MB in size. * Chunks up to 100 MB in size are now enabled for: - Amazon Cloud Drive (experimental) - Amazon S3 - Box - Dropbox - Google Cloud Storage - Google Drive - Microsoft Azure Storage - OneDrive (experimental) * Added a new checksumming algorithm and chunk I/O pipeline that supports partial read checksumming of read unit sizes. This works together with large chunk sizes (> 1 MB) and partial reads (for providers that support that) and allows for checksumming to take place with large chunks. - This checksumming algorithm (CRC32-Block) supersedes the old (CRC32) algorithm and will be used for all new drives by default. - The old algorithm is still supported and will work transparently for older drives (that were limited to 1 MB chunks). * [Issue #22850] Fixed OAuth 2.0 sanity check issue introduced with some changes recently. .419 * Added a new experimental OneDrive V2 provider that uses a newer API and supports chunks up to 100 MB. This provider is not backwards compatible with the V1 provider. .415 * Most cloud providers will now start off with 2 read threads and 2 write threads. You can raise this limit after creating a new drive. Most non-enterprise cloud providers are now limited to a maximum of 20 read/write threads and enterprise providers are limited to a maximum of 100 read/write threads. .413 * Changed the way that the CloudDrive.Service.exe.config file is deployed. * Increased the default HTTP connection timeout to 100 s. The HTTP connection timeout controls whether a HTTP request will be aborted because it took too long. The timeout is calculated from the time that the request headers are sent until the full response from the server is received. .410 * Increased the bandwidth display average to 30 s. * Decreased the default Read / Write threads for Google Drive to 2 / 2. * Enabled partial chunk downloads for Google Drive for chunk sized > 1 MB. * Enabled chunk sizes up to 100 MB for Google Drive. * Checksumming is now disabled after some predefined limit, and each provider can have its own limit. Checksumming requires full chunk downloads and so partial chunk downloading is always disabled when checksumming is used. * Added the ability for cloud providers to support arbitrarily sized chunk sizes. * Google Drive returns 403 error codes with detailed JSON objects describing the exact nature of the error, and they don't always mean that there is a security error. A lot of times they are throttling responses and are now treated as such. .409 * Fixed UI glitch that would cause drives to disappear from the credential list. * [Issue #21843] Fixed reauthorization. Multiple issues prevented reauthorization from working properly. * [Issue #21843] Fixed OAuth 2.0 for tokens that never expire (e.g. Dropbox). .408 * [Issue #15660] Added a Google Drive provider. * Added a unified architecture that all HTTP based providers can use. - Pluggable authentication architecture: - Support for OAuth 2.0 out of the box. - Concise HTTP interaction code with: - Automatic exponential backoff, when being throttled. - Centralized error handling. - Pluggable support for special authentication failures, such as OAuth 2.0 token expirations, and resolution to such failures. - Support for REST: - POST (multipart / URL encoded) with JSON support. - GET JSON. - Easily extensible to support future formats. .405 * [D] [Issue #20847] Fixed race condition on shutdown. Purge all existing I/O on the disk device prior to tearing it down. .404 * Increased IoManager_HttpReadWriteTimeoutMS to 40 S to deal with slower connections. * [Issue #20846] For OneDrive for Business, if the user enters the full URL into the "Tenant" or "Site Name" text boxes, extract the correct information from the full URL. 1.0.0.403 --------- * Added bandwidth limits to Amazon Cloud Drive. .402 * [Issue #20839] Amazon Cloud Drive can return negative values for the remaining free space. * Persist connection names across reauthorizations, where the name is just for display purposes. .401 * Fixed re-authorization upgrade issue. .400 * [D] Changed the criteria for when accelerated cache trimming kicks in. This allows for local write speeds to approach about 5 GB/s and still not overrun the cache. - >= 50 GB free space: Cache is trimmed every 10 seconds. (theoretical maximum write speed 5 GB/s) - < 50 GB free space: Cache will be trimmed every 1 second. (- < 6 GB free space: Write throttling begins.) - < 5 GB free space: All data will be unpinned, and cache will be trimmed every 1 second to maintain 5 GB free, if possible. (theoretical maximum write speed 6 GB/s) * Tweaked user feedback for write throttling. * Treat HTTP error 500 as a throttling response for Dropbox. .399 * Fixed various issues with the connect to Local Disk UI. The list of drives will now update as drives are added / removed without reopening the window. * [Issue #18739] Do not allow the cache to be placed on a virtual disk (VHD). * The UI requires administrative access to perform OAuth for providers that require a local HTTP callback endpoint. (only Google Cloud Storage for now) .398 * Treat Dropbox 503 errors as throttling. * Reduced the default number of read / write threads to 4 for Dropbox. .397 * OAuth secrets that have more than one instance are now synchronized, loaded and saved properly using a sigleton. This was causing random deauthorization of drives and connections for providers that expire their refresh tokens. This change completely overhauls how OAuth secrets are stored and used, and secrets stored in the old format will be automatically upgraded to use the new singleton model. Legacy secrets, stored in the credential store, will also be upgraded. .396 * When a checksum check fails on a chunk, never read from the in-memory cache on retry. This was causing perpetual checksum failures after the first checksum failure, with no chance of recovery. * Added workarounds to the Box provider in order to work around issues with the Box API and back-end. .395 * [D] Fixed system crash on drive detach / destroy. .394 * When then OAuth grant type is a refresh token, if a new refresh token is not given to us, do not revoke the previous one and assume it's still valid. This was de-authorizing Google Cloud Storage connections, but may have affected other providers as well. .393 * Fixed "To upload" ETA. Now using a 30 second upload speed average. * Fixed I/O threads window not showing accurate progress and response time. .392 * Added save to .PNG option for credential backups. * Added save to .PDF option for credential backups. * Added save to .TXT option for credential backups. * Fixed print margins on encryption credential backups. * Added a consistent web-UI experience for OAuth. * Completely got rid of the dependency on the Google SDK. Rewrote it from scratch, including OAuth 2.0. * Implemented local OAuth 2.0 callback web server for authorization. - Used for: - Google Cloud Storage (required) - Dropbox (with fallback to Web on failure) * Optimized UI handling of a large number of drives. * Turned off character ellipsis for credentials / drives in the new drives list. They were causing CPU issues and perpetual layout passes. * Simplified and unified the "Connect..." UI. - OAuth secret codes will now automatically populate into the UI, with a manual option if something goes wrong. - Connection names will now be auto-generated where possible by the provider. - Where we can't generate the connection names, they will default to generic names. - When re-authorizing an existing drive, a connection name is never asked for. .381 * [Issue #20809] Fixed issue with maximum read thread count. .380 * [Issue #20798] Amazon Cloud Drive now has a maximum limit of 2 upload and 2 download threads until the load issues can be resolved by Amazon. * [Issue #20798] Added a warning badge on the drive tab when using an experimental provider. * [Issue #20798] Added a warning when creating a drive on an experimental provider. * [Issue #20798] Marked the Amazon Cloud Drive provider (V2) as hidden and marked the original Amazon Cloud Drive provider as experimental. Since Amazon revoked our production status, both providers are non-functional at this time. We are waiting on Amazon to upgrade their infrastructure to support the kind of load that our program is generating. .379 * [Issue #20790] Experimental providers can now be shown in the provider list by enabling an option in the Troubleshooting menu. * [Issue #20790] Fixed mounting of experimental providers. .378 * Optimized memory usage of the Amazon Cloud Drive provider. * When Amazon Cloud Drive returns a 404 Not Found error, do not assume that the file doesn't exist. Amazon returns 404 codes when permission to a file is denied as well. * Prefix all data uploaded to Amazon Cloud Drive with a (null). * Encrypt all data uploaded to Amazon Cloud Drive regardless of whether full drive encryption is enabled. When full drive encryption is disabled, all the data uploaded to the Amazon Cloud Drive will be encrypted with a static AES key. This is not done for security, but to prevent Amazon's system from attempting to identify the types of files that we're uploading, based on pattern recognition. They do this regardless of what we're actually telling them in the upload request. This causes problems because Amazon categorizes images / videos / documents differently, and among other things performs some post processing on them, and then denies us access to the original content. - In order to implement this change, we need to have a new Amazon Cloud Drive provider (V2) that is not backwards compatible with the old one. - If you've created a cloud drive using the old (V1) provider, it will still mount using the old provider and will be marked as "deprecated". You should migrate your data to the new provider because the old (V1) provider uses developer keys, and thus is subject to strict throttling by Amazon. - All new Amazon Cloud Drives will be created using the new (V2) provider which uses production keys. - It is possible to create a cloud drive using the old Amazon Cloud Drive (V1) provider by enabling experimental providers in advanced settings. * Amazon has approved StableBit CloudDrive for production use. Switched to production keys. .371 * Fixed "Access denied" and "Invalid user name or password" errors in the File Share provider. .370 * Added retry logic to settings saving code. .368 * Enabled direct, non-buffered, cache to provider I/O when checksumming and upload verification are disabled. .366 * Increased internal buffer for Local Disk / File Share providers to 1 MB per thread. .364 * Fixed lockup with the File Share and Local Disk providers. * [D] Fixed edge case in write throttling code. * Added new "Background I/O" option under I/O performance, and enabled by default. All write threads will now read from the local cache at Background I/O priority by default. .355 * [D] Increased the virtual storage adapter's maximum transfer length. .352 * [D] Added cache manger deadlock resolution code for writes. * Compute upload ETA based on 10 second upload speed average. * I/O retry errors will no longer be shown in user feedback unless explicitly enabled under the troubleshooting menu. .346 * Fixed OneDrive for Business thread contention issue that was causing a performance bottleneck for read threads. * Added I/O thread troubleshooting window. .336 * Amazon Cloud Drive will continue to have issues until Amazon approves us for production use. .333 * Redirected licensing back at the production site. .332 * Attaching an existing cloud drive would cause a mount failure. .331 * Licensing fixes. .330 * Fixed updating issue introduced in build 319. .325 * Reworked how OneDrive for Business locates data chunks (now using OData's $filter). This should improve read performance for larger cloud drives. .324 * [D] Enabled cache manager over the cloud part cache, improving cloud drive write performance. .321 * Amazon Cloud Drive returns HTTP 500 errors when throttling. .320 * Volume free space statistics weren't being updated. * Increased default write abort timeout to 5 minutes to work better with slower upload speeds. * [D] Fixed deadlock with the cache manager. This was particularly an issue on slower systems and when write throttling was in effect due to low disk space. * Fixed spurious locking error in the Local and File Share providers. * [D] New caching behavior when running out of disk space on your local cache drive: - <--- Full Speed ---| 6 GB | --- Stage 1 (10 MB/s to 1 MB/s) --- | 5 GB | --- Stage 2 (1 MB/s to 10 KB/s) --- | 4 GB | --- Writes Failed ---> - At 6 GB free space, the writes to the cloud drive are throttled to 10 MB/s maximum, getting progressively slower as free space runs out. Pinning new metadata is not allowed at this point, but the cache does not start to trim yet. - At 5 GB free space, writes are throttled to approximately 1 MB/s maximum and all pinned data is unpinned. The cache starts to aggressively trim in order to maintain at least 5 GB free. - If all cached data is trimmed, and the free space continues to decrease below 5 GB, writes get progressively slower to 10 KB/s at 4 GB free. - < 4 GB free, all writes are failed. * Converted driver code to C++ due to a bug in Visual Studio 2015 that affects C code. (https://connect.microsoft.com/VisualStudio/feedback/details/1623363/error-command-line-error-exception-handling-option-can-be-used-only-when-compiling-c) * Fixed window style for < Windows 10. .319 * Fixed maximized window on x86 machines. * [D] Windows 10 support. - Created new driver binary targeting Windows 10. - Recompiled everything using Visual Studio 2015. - Implemented alternate window management code for UI glitch in Windows 10 that leaves 7 unusable pixels at the top of a border-less window. - Modified code to enabled DWM animations. .311 * [Issue #17718] When the digit grouping symbol was a " ", the drive size was not validated properly. .310 * [Issue #17739] File share I/O provider was not handling reauthorization properly. .309 * [D] [Issue #15661] Added workaround for a bug in Bitdefender Antivirus 2015 (trufos.sys). Details: https://stablebit.com/Admin/IssueAnalysis/15661 * [D] [Issue #15661] Cloud drives will now report a disk serial number. * [D] [Issue #15661] Drive sector size was always 4096 bytes. .308 * Updated WiX Toolset to 3.10 RC. * [Issue #17735] Errors from legacy code accessing the credential store are now recorded in error reports and ignored. .307 * [D] Decreased cache write throttling when running out of disk space (< 5 GB free). .304 * [D] Fixed crash on shutdown (APC_INDEX_MISMATCH). This had a high chance of occurrence if metadata pinning was taking place while the cloud drive was shutting down. There was a small race condition between when the cloud part would shut down and when the PNP physical disk would be removed. Because metadata pinning accesses the cloud drive directly, not through a file system, it had a high chance of triggering this crash. .299 * Tweaked Amazon S3. * Object Space provider is now experimental. .298 * [Issue #17711] Do not allow the cache to be stored on a StorLib device. .297 * Fixed secret retrieval crash on service start. .296 * For Amazon Cloud Drive, do not move individual chunks to the trash. * [Issue #17673] Fixed OneDrive for Business provider. * [Issue #16690] Added new Object Space provider. * Settings and drive options would not save to disk sporadically. * [Issue #17679] Upload verification should be disabled for the local provider by default. * [Issue #17679] Set the file pointer properly when the low order bytes of the offset corresponds to 0xFFFFFFFFF. * [Issue #17679] When constructing a large memory mapped buffer for checksum verification, ensure that the size of the buffer doesn't overflow. * [D] [Issue #17679] When stopping a cloud part, stop the cache first before the read / write / pin maps. * [Issue #17677] The number of storage chunks in use was sometimes being calculated incorrectly. * [Issue #17677] Storage chunks larger than 1GB were not working properly. * Enabled HTTP compression by default (IoManager_UseHttpCompression). * [Issue #16692] Changed how a provider's secret is stored. We no longer store it in the Windows credential store, instead we use DPAPI to protect it and store it locally on the drive. The credential store can't handle secrets > 2560 bytes which can cause issues with some providers. This change is backwards compatible. * Do not delete chunks when writing all zeros to Amazon Cloud Drive. 1.0.0.283 --------- * [D] [Issue #15663] Fixed crash if the disk is being queried for geometry information after the cloud part is destroyed. 1.0.0.282 --------- * Added ProviderRegistry_IncludeExperimental .config setting to allow the use of providers that are still experimental (just OneDrive for now). * Enabled upload verification by default on all cloud drives. * Fixed Amazon Cloud Drive file ID lookup issues. * For Amazon Cloud Drive, Box, OneDrive, if an upload is aborted, purge the file id cache for that chunk. * [D] Do not remove prefetching blocks from the prefetch map. * Changed read timeout / abort rules: - HTTP connections time out after 10 seconds. - HTTP download / upload operations time out after 10 seconds. - All read / write I/O is retried 10 times. - Stuck I/O is forcefully aborted after 00:01:55. - A hard I/O timeout in the kernel happens after 00:02:00. * [D] When a hard timeout occurs on a prefetch read request in the driver, notify the prefetcher that the read request failed. * [D] When prefetching fails due to a provider error, do not mark the requested range as successfully allocated in the cache. * Increased update check interval to 1 day for the public BETA. * Removed OneDrive because of their excessive upload throttling making usage infeasible. * [D] Fixed pool (memory) corruption issue. * Local / File Share providers now default to 1 write thread and 2 read threads to prevent disk thrashing. * No prefetching for local / file share providers by default. * I/O manager threads will now be forcefully aborted if they run for too long. (See IoManager_ReadAbort and IoManager_WriteAbort) * [D] Do not prefetch when low on disk space (< 5GB free). * For local I/O, after some period of time, release all local handles to the cloud part data (controlled by LocalIo_ReleaseHandlesDelayMS). * Added Amazon Cloud Drive provider. * Added File Share provider. * [D] Do not have the disk PDO hold a reference to the cloud part. This fixes a lockup on service stop if something has a direct handle open to the drive. * [D] Updated code signing certificate. * UI Fixes. * [D] Stability fixes to service and driver. * Added local drive provider. * [D] Implemented local cache chunking. This raises the maximum cloud drive size to 1 PB and the maximum volume size to 256 TB (NTFS limit). Local cache is no longer limited to 500 GB, it can be as large as the cloud drive. * Rewrote and restyled many windows using Logic. * Added key backup support. * [D] Added full disk encryption support (big update). * Added WSS integration for clicking links and a fallback method for when the browser is not properly registered on non-WSS. * Selected UI state is now saved and synchronized. * Added remote control back-end. * Added tool tips to the new drive tab. * Cloud free space is now checked on a resize and a create, for providers that support it. * Added Dropbox support. * Centralized OAuth handling. * Added Box support. * Added TRIM support. Data that has been deallocated by the file system will be removed from the upload queue. * All new drives are now GPT by default. * Clean I/O manager disposal. * [D] Purge the control device queue when the disk gets destroyed. * [D] Verified driver against memory leaks and fixed a few. * Added new "Cleanup" task. This is/was necessary when shrinking an existing drive and was previously part of resizing. But this way, resizing is very fast and you can defer cleanup until a later time. * [D] Online drive resizing. * Added automatic updates. * Pin directory index metadata. * Optimized pinning to 1 byte resolution. * Pinning file system metadata now pins the entire MFT and its non-resident attributes. * Cache will not be flushed by writes. * New writes will be flushed first. * Write throttling will start if the local drive has < 1 GB free. - At 1 GB free, writes are throttled to 1 MB/s. - Also, at 1 GB free, the cache will begin to trim in an attempt to keep the disk space at 1 GB free. - Writes will get progressively slower as the free space diminishes. - Approaching 250 MB free, write speeds approach 512 B/s. - User is informed if throttling is occurring. - At 250 MB free, writes begin to fail. - User is then warned with a more dire message. * UI optimizations and bug fixes.