Science confirmed: Text replacements do not sync


#1

Originally posted on the MacStadium blog:

Text replacement is a useful tool that can be used on macOS and iOS. With this feature you create shortcut text that, when typed, expands to something longer.

For instance, you might create a shortcut so that “4yum” automatically expands to “For the best pastries in the world, visit Les Madeleines in Salt Lake City and ask for a Kouing Aman. Your life will change.” (What? It’s the truth. Highly recommended.)

You can also use it to force an autocorrect. For instance, I make “welll” automatically go to “we’ll” to save the apostrophe hunting.

Once setup, the feature is incredibly useful.

There are fantastic third party apps like Textexpander that have been doing this for years. While I still love Textexpander on my Mac, it’s much less convenient on an iPhone because it requires a 3rd party keyboard. Switching back and forth between keyboards kill the convenience so I don’t use it.

Apple introduced text expansion as a built-in feature with Snow Leopard (2009) and iOS 5 (2011) updates.

In iOS 6 (2012), syncing between Mac and iOS devices was introduced. At least that is when Apple said it was introduced. I have yet to see the feature roll out. Therein lies the premise of this post.

Text replacement syncing is completely broken. Sometimes it works, sometimes it doesn’t. Sometimes it will only sync back old snippets that you have deleted. Sometimes the sync will work one direction, but not the other. Every time I ask about this on Twitter, it brings a strong response of similar experiences.

Even though I know it’s broken, I decided to get scientific about it. I felt like I had a unique setup to test well for a few reasons:

  • Like many of you, I have a good number of personal Macs and iOS devices that I use every day.
  • I have four kids with iPods that I can commandeer at any time “for science.” This allows me to test various iOS installs.
  • I have hundreds of Mac minis available at my disposal so I can test all sorts of OS X and macOS releases, including clean installs and upgrades.

Now, I’m not a scientist but I do play one on this blog so here was my approach:

My personal iPhone is my control device. It’s the one I use every day and is the most important to have all of my text replacement snippets. It’s the only one that has the full list of all snippets because it is where they were all added.

On every other device, I created only one single snippet that was self identifying. For instance, on an 6th generation iPod running iOS 10 I’d create “*6iPodiOS10” which would expand to “6th generation iPod running iOS 10 created on Aug 15”, etc. This way I would know what was syncing from where.

I turned on iCloud Drive and syncing of any kind that could be found in the Settings.

I began my observations on July 26 with the release of iOS 11 beta 3. I installed that beta on my personal iPhone and iPad and deleted everything on all other devices.

I checked all of the devices every 6 days. In a general observation, anything to sync would take 2-5 days. (Sidenote: WHY DOES SYNCING TAKE SO LONG WITH THIS STUFF?!?)

I won’t overload you with the screenshots. Nearly all of them had the single entry, and when things do sync, there’s too much personal info there for me to blot out every time. (ie, address, phone numbers, school addresses, etc)

With that being said, here is the graph showing the state of each device when checked in ten 6-day periods:

I understand the details on the graph are limited. The general idea is that syncing text replacements is a complete mess.

The iPhone 7 and a clean installed iPad Pro stayed in sync nearly the whole time. Adding a snippet to either device would sync immediately.

Period 3 for the Macbook was a glorious day. It’s the lone surprise green. To get that result I went into the Finder and deleted the following files:

~/Library/KeyboardServices/TextReplacements.db
~/Library/KeyboardServices/TextReplacements.db-shm
~/Library/KeyboardServices/TextReplacements.db-wal
~/Library/Group Containers/com.apple.InputMethodKit.TextReplacementService.

It was a total shot in the dark. After that, it synced.

Then the other two iMacs kind of synced. (Showing only the snippet from the MacBook, but not the ones from the iPhone.)

Then all of them went back to where they were at the beginning. What a let down.

When the 10.13 GM was released I put it on a Mac mini it shook some things up for a few Macs, but it was just random cases of sync. Hopefully future updates of macOS 10.13 will show more hope.

Conclusion

From my own experience, syncing of all other data via iCloud has really improved. Notes, Calendar, address book, reminders, photos, etc all sync almost instantly across all devices.

What is so special/not special about Text Replacement snippets that makes it so hard?

I do wonder if some if this comes from my main machines going through years and years of upgrades. For instance, the iMac 5K hasn’t had a clean install of the OS in over a decade. I’ve always upgraded to new releases, and cloned over to new Macs.

I wonder if someone went out to buy a brand new Mac and iPhone today with no history of using iCloud or syncing, would their text snippets come through fine?

For anyone who has years of data like me (and probably you), it seems like there would need to be some way to mark the master list of snippets. It would probably need to be on iCloud.com where it could show you what all of your devices are syncing and you can choose the one to replace all the others.

I have no idea if there is a fix for text replacement syncing. If there is one, I’d be happy to hear about it on our forums or on twitter. I wish I had a fix to share. I mostly put forward this data to show just how random and far reaching the issue is on Apple devices.

(It’s fixed! See the update below.)


#2

An interesting update (not yet posted to the blog entry, but putting it here for discussion):

This morning, I spun up a Mac mini in the data center with a brand new install of High Sierra. I enabled iCloud Drive and of course nothing would sync from the iPhone master list.

However, any additions that I made on this brand new Mac mini were recognized on nearly every device. If I added a snippet there, it went to all the iOS devices, Macbook, iMacs, etc.

Perhaps a clean install of High Sierra is now saving snippets correctly? It’s a start I guess. But who knows if it will still be working this afternoon.

So at a minimum, it seems you can start a brand new master list to manage all the other devices. It’s also pretty convenient because snippets on a Mac are simple to back up. Just highlight them all, drag to the desktop, and a plist will be created.


#3

I have been struggling with this since iOS 8 launched.
If my only problem was that shortcuts didn’t sync between devices, I would’ve been relatively thrilled. But in my case I had some very bothersome shortcuts that would interfere with typing that would keep coming back no matter how many times I deleted them.

I tried all kinds of things to rectify this situation - even going “full nuclear” and resetting ALL of my devices without a restore. But even that didn’t solve this problem - eventually the zombie shortcuts would come back. There was obviously something bewildering happening with my iCloud account.

After some digging, I discovered that there were some old files left over from the CoreData sync that had been (and maybe still is?) responsible for syncing keyboard shortcuts. On a jailbroken iOS device (this may have been iOS 9 days still, at the time) I found the following:

In /var/mobile/Library/Keyboard/CoreDataUbiquitySupport/

this directory contained seven subdirectories with names like:
mobile~8E030ECA-E4E3-5563-91A5-65C2AFF8E6B0

Within each of those is a directory tree that contained subdirectories like
local/store, and many of those contain sqlite databases called
CloudUserDictionary.sqlite

By investigating the database tables like ZUSERDICTIONARYENTRY contained within those sqlite files, I could see many old, deleted shortcuts in there – in many cases they were in there dozens of times — probably the number of times they have been deleted!

By manually deleting these directories from my iOS devices, then manually deleting all the keyboard shortcuts, I was able to purge the zombie shortcuts. After this, setting up a freshly restored iOS device would sync only the keyboard shortcuts I wanted (or maybe not always, but at least it wouldn’t sync the unwanted ones that were supposed to be deleted!).

The following blog post (including a lengthy comment from me with some of the above info) details the experiences of several people fighting with this particular Apple/iOS/CoreData defect:

I’m not sure if any of this information helps you get any of this working - even in my case after all I messed with this stuff, the best I could do is get pairs of devices syncing with each other but all four of my devices (Two Macs, iPhone, and iPad) wouldn’t ever totally be in sync.

This has been a complete and utter gong-show for years and I just wish Apple would fix this and restore some sanity to text replacement syncing!


#4

Lots of helpful posts to put a spotlight on the issue:
https://512pixels.net/2017/09/iclouds-text-expansion-syncing-is-busted/




#5

A pleasant surprise from John at Daring Fireball:


#6

I just tested text-replacement shortcut syncing on iOS 11.1 Beta and macOS 10.13.1 beta which dropped today.
IT FINALLY WORKS!!!

This fixed it for me, at least. If you’re able to run developer betas, try that. It might finally get things working (at least for devices running the .1 installs.


#7

UPDATE:
We aren’t quite there yet. After the update, my MacBook and iPad are syncing flawlessly, but my iPhone wasn’t syncing at all. On my iPhone I logged out of iCloud and did a “reset all settings”, then logged back in and set up my phone again. After this, all my keyboard shortcuts did sync with iCloud the once, but they don’t continuously stay in sync when anything is changed.

Also, I just noticed that the shortcuts that synced included a bunch that I had deleted yesterday. So this is all still a huge mess.

Sigh… maybe the final version will fix this. At least for now I got my shortcuts back, although that was a major pain.

UPDATE 2:
I just tested changing text replacement shortcuts on my iPhone 6s Plus, iPad Air, iPad Pro 12.9 and MacBook Pro 15" and they are all syncing on the latest iOS/MacOS developer beta despite it not working everywhere initially. This is great news. Perhaps it just needs time to shake out?

The only thing I have that isn’t syncing now is a Hackintosh that is running High Sierra 10.13.0 - so this is a very good sign that Apple has finally, mercifully fixed this problem in the .1 releases. Fingers crossed!


#8

It’s refreshing to see this bother others as much as it does me. I have wasted an extreme amount of time on this issue - I’m somewhat embarrassed to say it was probably more time than what would have been saved even if it worked flawlessly - but sometimes if you get fixated on a problem it’s not possible to ‘turn it off’

About 10 months ago I put in a ticket with Apple, they deleted my keyboard dictionary and it worked flawlessly across all my devices for about 3 months. Then I logged into a backup account on the same system, which was synced with my apple ID (which I hadn’t logged into since prior to the ticket 10 months ago) and whatever was in that account synced and it both broke syncing across some (but not all) of the devices AND brought back items which were deleted. I created another ticket, they read the past ticket notes and escalated again and i was told ‘the issue you’re experiencing is still under investigation and is expected to be resolved in a future update.’

Just a bit more waiting I suppose.

In the mean time, a few weeks ago i noticed ‘TextInput’ / ‘profile’ / ‘Secure’ in my iCloud storage. Until I read the recent updates this was on my to do list:
On all devices with text replacements:
Delete text input in icloud drive
Disable icloud drive
Reset keyboard settings
Enable icloud drive
add items one by one and test if they sync…

now that I see apple is changing the mechanism… i suppose I’ll hold off…


#9

Here we are on 27 November and syncing is working immediately across all devices running High Sierra and iOS 11. It doesn’t matter if the Mac/iPhone/iPad is a clean install of the OS, or has been upgraded 10 times over ten years. I think it’s safe to say that it’s officially been fixed.

I’ve now made a master list of text replacements on one device and it syncs to all other devices. Also, be sure to highlight all and drag to the desktop for a good backup.


#10

Hi there, coming here because I’ve had the same issue syncing TR across a number of different iPhones, all running iOS 11.2.2.

When I spoke with Apple, they told me the only way to sync text replacements across phones was to back up phone 1 to iTunes and restore that backup onto phone 2 to get the same text replacements on phone 2. I did that, however, and while everything else was restored to phone 2, the text replacements were not.

If I’m understanding this thread correctly, it sounds like the text replacements might sync if I backup phone 1 to iCloud and then restore the iCloud backup to phone 2, instead. Is that correct? Please provide any tips possible. Thanks!