DTTG, Synology and WebDav Puzzle.... Seeking Inspiration

DEVONthink Scenarios
1

I am attempting to sync my DevonThink (V 3.85) databases between my Mac(s); iPad; and iPhone using WebDav running on a Synology DS220+.

I have read and followed an excellent thread found in the forum (How to use Synology NAS as sync store). I am unable to access the WebDav server and sync when outside the LAN.

To be clear, I have set up an “internal” sync on DTTG with the following scheme:

URL: https://10.0.0.XX:5006/dtsync/
(Where dtsync is the name of the sync store in the Synology service.)

User Name: XXXXXX
Password: XXXXXXXXXX

The above works perfectly across all databases while on the LAN.

I have set up a second, identical, DTTG sync service with the exception of the URL:

https://quickconnect.to/XXXXXplacenas:5006/dtsync

The quickconnect.to address shows "normal in the Synology Control Panel but no joy when attempting access outside the LAN via another Wi-Fi and/or cellular network. (In fact, this set up isn’t working even inside the LAN.)

My NetGear router table is set to forward ports 5005-5006 to the same ports to the internal IP address set up for the Synology NAS on the network.

To troubleshoot if the problem(s) are in some way caused by the quickconnect.to framework, I have set up a XXXXXX.synology.me URL (which also has a Normal Status in Control Panel). Neither configuration allows sync.

Performing the sync using this configuration I “see” the various database names in DTTG but no sync takes place. (S0 I “assume” there is at least a handshake connection to get the names.)

The error I am receiving in the log currently is , “Invalid Encryption Key.” To my knowledge, there is no encryption key attached to this sync store (but there was a key I used when using iCloud sync between these devices some time ago. This sync method proved very unreliable which is why I moved to WebDav to begin with).

I am fine with no encryption key as I am the only one accessing this data and it isn’t particularly sensitive or useful to others. Can I clear any possible encryption key associated with the store?

As part of my troubleshooting, I did find a key in the KeyChain Application, (DEVONcloudy Encryption (WebDAV)) on the Mac I have with me (running 12.4) and tried copying/pasting this key into the DTTG sync configuration. This also didn’t resolve the problem and I have since cleared it from the set up on my iPad. The error in the DT log persists…

Since I can sync while internal to the LAN, I am doubly confused by this error message as I have no encryption key entered in this DTTG configuration (that is viewable or known to me).

I am able to sync devices across the internal LAN and have multiple backups of my databases via Time Machine, CCC, and BackBlaze so with proper care, I am willing to take steps to rebuild the Sync Store if that is a useful recourse. I don’t think the store itself is corrupt since sync is possible with the internal URL.

I have exhausted the troubleshooting steps which I am able to research and explore. If this is purely a Synology/port forwarding (NetGear RAX 75 router) issue, I will reach out on forums more focused to these devices. I thought I would start here and see if log files, or someone may have an inspiration worth exploring.

Thanks in advance for any thoughts, help, or assistance with this issue. I hope you have a wonderful day!

2

You can’t use quickconnect for that. You must use a DynDNS address like xxx.synology.me. I’m fairly certain that the procedure is described in the post you linked to.

And please turn off port 5005 and HTTP.

Good to the first point, the second one is irrelevant. What would be relevant is: Can you connect to this location using finder by “connect to server” and then entering “https://xxx.synology.me:5006”?
As an aside: do you have a certificate for this domain?

The problem is certainly located somewhere in your network setup. But since your post contains precious little facts on that aspect, I don’t really know. Making the server show up in Finder, as I said before, is a first step. Also, it would help if you’d tell if you’re connecting to the synology.me location from inside your network or from outside it. Port forwarding is not relevant if you’re still sitting inside your network. But if you’re on the outside (e.g. on your iPhone/iPad with a 4/5G connection and Wifi turned off – what does entering
https://xxx.synology.me:5006
in the browser tell you?

3

Thanks for the response! I apologize for not providing more details regarding the actual network. I have tried a couple of things based on your comments. I removed port 5005 from the forwarding rule.

I swapped the xxxx. synology.me URL for the QuickConnect version and,

I tried connecting to xxx.synology.me:5006 via Finder (No joy.)

I haven’t gotten outside the network to try accessing via a browser but will later today. Also, it appears a certificate has been issued from Let’s Encrypt through Synology.

I agree with your assessment. This must be an issue relating to my network set up (although it isn’t overly complex in design). I will try accessing from outside “the wire” and see what happens but I don’t hold much hope.

Thanks again for your thoughts and time!

4

Well, “no joy” conveys that something didn’t work. But what exactly? Did the connection time out, did you see an error message (which one)? It’s very difficult to get to the core of a problem without sufficient information. Also, did you try to connect to https://xxx.synology.me:5006 or did you really leave out the protocol part (yes, I sound like a pedantic fart, but unless the information is complete, it’s very difficult to get to the core of a problem – wait, I said that already).

Simple things to try in the terminal:

  • nslookup xxx.synology.me => what do you see?
  • if that doesn’t give you an error:
    traceroute xxx.synology.me => what do you see?

If the first command does not work, your DynDNS setting is not working. I can’t tell you why.
The second command should show the network path from your machine to the router and then immediately to the NAS (provided it’s connected directly to the router). Is that the case? And please do not edit out the IP addresses in the traceroute output: they’re private anyway, so completely useless outside your private network.

And this certificate is

  • issued for the same subdomain xxx.synology.me that you’re trying to connect to?
  • not expired?
5

Sorry for the delay in updating this thread. Family issues kept me from working on this issue for the past few days. I would like to thank everyone for their ideas and assistance. I especially thank @chrllek for his insight and guiding me towards the solution.

I believe I have largely resolved the issue (but won’t close the book until this works reliably for a week or so) and it isn’t quite doing so yet. The problem does not appear to originate with DT or my Synology NAS.

My network configuration begins with a fiber Gateway (AT&T BGW 210-700). There is a NetGear Night NightHawk RX-75 Router hardwired to the Gateway. The BGW does not have a bridge mode available but there is an IP Passthrough configuration which provides much of the functionality of a bridge if properly configured. Gaining this understanding has taken more than a moment’s research.

While we were away from the house this past year, another NightHawk router failed (due to a power surge, storm, or old age, I will never know). I had a friend install the replacement router while I was out of the country with me providing remote guidance. He did a great job and by using the Wi-Fi SSIDs from the prior router virtually all the network attached devices reconnected without issue or further intervention.

I neglected to remember that the Gateway’s IP Passthrough configuration relies upon the router’s MAC address which of course changed when the router was swapped out! Failure to update this in the ATT Gateway configuration stopped the passthrough from working properly (although for reasons I still don’t understand and am not going to spend time pondering the two devices did work seamlessly save for the port forwarding issues resulting in this thread).

In any event, updating the MAC address for the NetGear router on the ATT Gateway appears to have resolved the basic problem of accessing the WEBDAV server on the Synology NAS from locations external to the LAN.

Here is the current lookup and trace route information:

randyXXXX@RGW-MB-Air ~ % nslookup xxxxxxxxxx.synology.me
Server: 10.0.0.1
Address: 10.0.0.1#53

Non-authoritative answer:
Name: xxxxxxxxxx.synology.me
Address: 86.XX.XX.199

randyXXXXX@RGW-MB-Air ~ % traceroute xxxxxxxxxx.synology.me
traceroute to xxxxxxxxxx.synology.me (86.XX.XX.199), 64 hops max, 52 byte packets
1 10.0.0.1 (10.0.0.1) 1.871 ms 1.010 ms 0.926 ms
2 192.168.1.254 (192.168.1.254) 2.052 ms 2.005 ms 2.189 ms
3 23-117-132-1.lightspeed.nsvltn.sbcglobal.net (23.117.132.1) 3.053 ms 3.311 ms 3.116 ms
4 99.174.25.192 (99.174.25.192) 3.258 ms 3.627 ms 3.486 ms
5 * * *
6 * * *
7 32.130.17.65 (32.130.17.65) 9.987 ms 10.295 ms 16.988 ms
8 192.205.36.158 (192.205.36.158) 9.670 ms 23.779 ms 9.269 ms
9 ae-5.r24.atlnga05.us.bb.gin.ntt.net (129.250.4.192) 9.371 ms 9.795 ms 10.134 ms
10 ae-4.r25.asbnva02.us.bb.gin.ntt.net (129.250.4.165) 30.905 ms 27.996 ms *
11 ae-7.r21.londen12.uk.bb.gin.ntt.net (129.250.2.110) 104.054 ms 105.345 ms 102.422 ms
12 ae-1.r20.londen12.uk.bb.gin.ntt.net (129.250.2.182) 103.669 ms 103.383 ms 101.696 ms
13 ae-11.r20.amstnl07.nl.bb.gin.ntt.net (129.250.5.2) 100.253 ms 100.503 ms 104.686 ms
14 ae-0.a01.amstnl07.nl.bb.gin.ntt.net (129.250.7.87) 116.371 ms 123.522 ms 114.085 ms
15 * * ae-0.kpn.amstnl09.nl.bb.gin.ntt.net (81.20.65.14) 104.647 ms
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
31 * * *
32 * * *
33 * * *
34 * * *
35 * * *
36 * * *
37 * * *
38 * * *
39 * * *
40 * * *
41 * * *
42 * * *
43 * * *
44 * * *
45 * * *
46 * * *
47 * * *
48 * * *
49 * * *
50 * * *
51 * * *
52 * * *
53 * * *
54 * * *
55 * * *
56 * * *
57 * * *
58 * * *
59 * * *
60 * * *
61 * * *
62 * * *
63 * * *
64 * * *
randyXXXXX@RGW-MB-Air ~ %

As you can see, it still appears to be timing out on at least some ping attempts. (The above is taken from within the LAN from my MB Air which at this location has my primary DT installation.)

The newest wrinkle involves the very slow syncing and inability to download any documents to the DTTG instance on the iPad with DTTG (this problem exists using the outside domain WEBDAV configuration whether accessing from within the LAN network or via an outside Wi-Fi source).The trace route above provides a fairly surefire concern.

I am sure there’s a connection between the timeout and the remaining sync issues but I don’t know if there’s anything more I can do on my end. The sync is slow or stopped BUT the log file on the DT side (which I have using the same synology.me URL rather than the internal 10.XX.XX.XX internal LAN address to the WEBDAV Sync Store) shows only this:

9/6/22, 4:31:46 PM: Inbox Couldn’t upload 1 pending files.
9/6/22, 4:31:46 PM: Inbox 1 items left to be uploaded.
9/6/22, 4:34:16 PM: Investment Ideas Couldn’t upload 4 pending files.
9/6/22, 4:34:16 PM: Investment Ideas 4 items left to be uploaded.

This issue should probably be a new thread but why would accessing the same sync store remotely (versus using the local IP within the LAN) even create the need to upload “documents” from DTTG (which has always been set to “On Demand” download)? And why can’t I download even a 1 KB document “On Demand?” from the sync store to the DTTG instance on the iPad?

I am aware that the document data on DTTG is more of a placeholder with a link/target to the actual document, but the upload progress suggests a massive delay (particularly when compared to the suync speeds within the LAN).

Based on the trace route, it appears to me that there still may be some connectivity issue(s) which are unresolved. Progress! Just not a full solution yet.

Thanks to all for their help and insight!!!

1 Like
6

Now I’ll try to reply to some of the issues here.

First off, I suppose that WebDAV sync is now working reliably in your local network.

So, everything that follows refers to a remote connection to your local WebDAV server.

traceroute output: The output does not necessarily show timeouts – there could be hosts that do not respond to ICMP packages. But it does show some weird (in my opinion) network topology. I don’t know where you’re located, but it seems to be somewhere in the KPN network in the Netherlands. OTOH, the first network hop outside your local network is 23.117.132.1, which is part of an AT&T network range, presumably in the US. If you’re on the KPN network with your WebDAV server (i.e. 86.xxx.199), why is your first hop on the AT&T network? If you’re on the AT&T network, why is your WebDAV server’s IP address not in their network, too?

In fact, your WebDAV server’s external IP address should be the same as your router’s external IP address. That’s why you need the port forwarding in the first place.

Also, you seem to be using two private local IP ranges (10.0.0.x and 192.168.1.x) – why is that? I fear that this might cause additions problems for port-forwarding, NAT etc.

I’m ignoring the sync questions: they belong in their own thread, but only after sync works reliably.

More questions to clarify things

  • what is your local network topology (internal IP addresses and connections between router and other devices, external IP address of the router vs. output of whois xxx.synology.me?
  • what is the geographical situation (NL vs. US)?
  • from your local network, what does traceroute -n xxx.synology.me tell you?
3 Likes
7

I have deleted the DDNS entry I mistakenly left public in the prior post. Thank you for pointing out this serious security breach!

I created a new XXX.synology.me entry. The new nslookup and traceroute results for this from a computer on the same network is below. I will try to find a way to attach a file detailing my network configuration at this location. Hopefully, this will shed some light on the situation.

Your comments about the prior traceroute showing European hops is interesting and puzzling. We have traveled to a European based region (in the Northern Carrabean) in the last six months and the MB Air I am running now made the trip. I have no idea why or how this could impact the trace route results. It has been rebooted several (many) times and gets all its network settings as a DHCP client (in this case of the local NetGear router).

I am currently in the US and connected via ATT Fiber so that is my Gateway.

In any event, after setting up the new synology.me DDNS, here are the results:

nslookup XXX.synology.me
Server: 10.0.0.1
Address: 10.0.0.1#53

Non-authoritative answer:
Name: XXX.synology.me
Address: 99.XX.40.9XXX

randyXXXX@RGW-MB-Air ~ % traceroute XXX.synology.me
traceroute to XXX.synology.me (XX.XX.40.XXX), 64 hops max, 52 byte packets
19-83-40-109.lightspeed.nsvltn.sbcglobal.net (99.XX.XX.109) 1.167 ms 1.226 ms 1.010 ms
randyXXXXXRGW-MB-Air ~ %
randyXXXX@RGW-MB-Air ~ %

randyXXXXXX@RGW-MB-Air ~ % traceroute -n XXX.synology.me
traceroute to XXXX.synology.me (99.XX.XX.109), 64 hops max, 52 byte packets
1 99.XX.XX.109 1.566 ms 0.966 ms 0.992 ms
randyXXXX@RGW-MB-Air ~ %

Looking at the WEBDAV logs directly from the Synology, it appears that the server is actively being queried by the user account I set up and download/upload and deletion activities are taking place but are not consistently reflecting on the DTTG or DT databases with remote access.

Syncs aren’t working reliably in either direction (I only have DTTG attempting access on my iPad and DT running on one box at the moment.) with the remote configuration. Again, fine substituting the internal IP for the Synology in the sync settings.

On demand document downloads for existing entries to DTTG are still not occurring. (I also cannot connect to the server through Finder’s Connect To Server using, XXXXX.Synology.Me:5006.)

Since there is connectivity occurring, this is probably a problem for a new thread. If I continue to pursue this issue. At this point, I have spent at least a couple days time attempting to resolve this matter and I am coming to the conclusion it all just may not be worth it.

Personally, I am frustrated because I am sure there has to be a logical explanation for the sync problem. (DTTG shows no “triangle” warning badge and the sync store does verify.) If the remote queries are reaching the WEB DAV server as the log suggests, it should work remotely as well as it does inside the LAN using the same sync store to my eyes…

I appreciate everyone’s thoughts and suggestions. I suspect I will continue to devote time researching this and trying various potential remedies as time allows.

8

Can you hold the Option key and choose Help > Report Bug to start a support ticket?
Thanks.

9

I just uploaded log files per your request. I appreciate your taking a look! (I wasn’t able to find a way to upload my local LAN topology to the forum. If it’s interesting or helpful, let me know how to provide it. I appreciate any insights you might provide to help unravel this problem/mystery!

10

Sorry, I’m slowly drowning in all that. But I seem to gather this much:

  • tracerouting to your synology.me server from your local net needs only one hop.
  • that should indicate that the router’s external IP address is the same as your synology.me server’s one (you didn’t post anything about that, though)

That “syncs aren’t working reliably … with the remote configuration” could be related to the weird routing shown in one of your earlier posts. If that persists (i.e. if traffic from your local, US network travels to Europe and back again), you should talk to your IP provider. Also, looking at the output of
ping <xxx>.synology.me
from outside your network (i.e. a host not on 192.168.xxx or 10.0.xxx) should give you some indication as to the reliability of the IP connection. In my case (using my iPhone on the mobile phone network to ping xxx.synology.me), I see an average of 30ms, and a worst of 32ms ping time with no packet loss. The same if I connect my MacBook to the iPhone using the private hotspot (though ping times get a bit worse, which is to be expected).

Actually, escaping the complete connect string with backquotes would have helped http://xxx.synology.me:5006 can of course not connect to your WebDAV server, since the port 5006 is for http_s_ – SECURE http. Telling finder to connect with http to a port where it will only find https can not work. Regardless of any other issues.