Repository navigation
BrowsertStack tests fail out of the box #24
Description
I'm running node_modules\.bin\intern-runner config=tests/intern.js (on Windows 7) from the complete-tutorial branch and the tests fail, first because an element isn't found, then because my account (free one for open source projects) can't run parallel sessions:
Tunnel download: 99.873%
Tunnel download: 100.000%
BrowserStackLocal v3.7
Ready
Tunnel started
‣ Created session internet explorer 10 on WIN8 (4cc4a26552d56c793fc462abf448f50aab4127fd)
✓ internet explorer 10 on WIN8 - unit tests - hello - greet (0.001s)
‣ Created session internet explorer 11 on WIN8 (71ef70bd5dee944f3da500b26eb4c86fee1d6361)
✓ internet explorer 11 on WIN8 - unit tests - hello - greet (0s)
× internet explorer 10 on WIN8 - index - greeting form (11.908s)
NoSuchElement: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session/4cc4a26552d56c793fc462abf448f50aab4127fd/element / {"using":"css selector","value":"body.loaded"}] Unable to find element with css selector == body.loadedRNING: The server did not provide any stacktrace information)
Command duration or timeout: 5.23 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.45.0', revision: '5017cb8', time: '2015-02-26 23:59:50'
System info: host: '172-16-4-215', ip: '172.16.4.215', os.name: 'windows', os.arch: 'x86', os.version: '6.2', java.version: '1.8.0_51'
Driver info: org.openqa.selenium.ie.InternetExplorerDriver
Capabilities [{browserAttachTimeout=0, enablePersistentHover=true, ie.forceCreateProcessApi=false, ie.usePerProcessProxy=false, ignoreZoomSetting=false, handlesAlerts=true, version=10, platform=WINDOWS, nativeEvents=true, ie.ensureClession=true, elementScrollBehavior=0, ie.browserCommandLineSwitches=, requireWindowFocus=false, browserName=internet explorer, initialBrowserUrl=http://localhost:9629/, takesScreenshot=true, javascriptEnabled=true, ignoreProtectedModeSngs=false, enableElementCacheCleanup=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=dismiss}]
Session ID: 1fcb3bbb-7f69-4794-b425-b7cf5df923da
*** Element info: {Using=css selector, value=body.loaded}
at ProxiedSession._post <node_modules\intern\node_modules\leadfoot\Session.js:59:30>
at ProxiedSession.Session.find <node_modules\intern\node_modules\leadfoot\Session.js:1055:15>
at Command.<anonymous> <node_modules\intern\node_modules\leadfoot\Command.js:42:36>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
at Command.find <node_modules\intern\node_modules\leadfoot\Command.js:23:10>
at Command.prototype.(anonymous function) [as findByCssSelector] <node_modules\intern\node_modules\leadfoot\lib\strategies.js:24:16>
at Test.registerSuite.greeting form [as test] <tests\functional\index.js:12:6>
at <node_modules\intern\lib\Test.js:211:24>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
=============================== Coverage summary ===============================
Statements : 100% ( 4/4 )
Branches : 100% ( 2/2 )
Functions : 100% ( 2/2 )
Lines : 100% ( 4/4 )
================================================================================
internet explorer 10 on WIN8: 1/2 tests failed
× internet explorer 11 on WIN8 - index - greeting form (11.777s)
NoSuchElement: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session/71ef70bd5dee944f3da500b26eb4c86fee1d6361/element / {"using":"css selector","value":"body.loaded"}] Unable to find element with css selector == body.loadedRNING: The server did not provide any stacktrace information)
Command duration or timeout: 5.12 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.45.0', revision: '5017cb8', time: '2015-02-26 23:59:50'
System info: host: '172-16-4-251', ip: '172.16.4.251', os.name: 'windows', os.arch: 'x86', os.version: '6.3', java.version: '1.8.0_51'
Driver info: org.openqa.selenium.ie.InternetExplorerDriver
Capabilities [{browserAttachTimeout=0, enablePersistentHover=true, ie.forceCreateProcessApi=false, ie.usePerProcessProxy=false, ignoreZoomSetting=false, handlesAlerts=true, version=11, platform=WINDOWS, nativeEvents=true, ie.ensureClession=true, elementScrollBehavior=0, ie.browserCommandLineSwitches=, requireWindowFocus=false, browserName=internet explorer, initialBrowserUrl=http://localhost:14371/, takesScreenshot=true, javascriptEnabled=true, ignoreProtectedModeings=false, enableElementCacheCleanup=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=dismiss}]
Session ID: ca5a6eb8-ce43-4625-bcc3-1ffe846402f3
*** Element info: {Using=css selector, value=body.loaded}
at ProxiedSession._post <node_modules\intern\node_modules\leadfoot\Session.js:59:30>
at ProxiedSession.Session.find <node_modules\intern\node_modules\leadfoot\Session.js:1055:15>
at Command.<anonymous> <node_modules\intern\node_modules\leadfoot\Command.js:42:36>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
at Command.find <node_modules\intern\node_modules\leadfoot\Command.js:23:10>
at Command.prototype.(anonymous function) [as findByCssSelector] <node_modules\intern\node_modules\leadfoot\lib\strategies.js:24:16>
at Test.registerSuite.greeting form [as test] <tests\functional\index.js:12:6>
at <node_modules\intern\lib\Test.js:211:24>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
=============================== Coverage summary ===============================
Statements : 100% ( 4/4 )
Branches : 100% ( 2/2 )
Functions : 100% ( 2/2 )
Lines : 100% ( 4/4 )
================================================================================
internet explorer 11 on WIN8: 1/2 tests failed
Suite internet explorer 9 on WINDOWS FAILED
UnknownError: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"internet explorer","version":"9","platform":"WINDOWS","browserstack.local":"true","browserstack.localIdentifier":"522680565","browserstack.selenium_version":"2.45.0","name":"tests/intern.js","idle-timeout":60}}] 1 session is currently being used. Please upgrade to add more parallel sessions.
at Server.createSession <node_modules\intern\node_modules\leadfoot\Server.js:307:15>
at <node_modules\intern\lib\executors\Runner.js:174:22>
at retry <node_modules\intern\lib\util.js:695:13>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
Suite firefox 37 on MAC FAILED
UnknownError: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"firefox","version":"37","platform":"MAC","browserstack.local":"true","browserstack.localIdentifier":"1438522680565rowserstack.selenium_version":"2.45.0","name":"tests/intern.js","idle-timeout":60}}] 1 session is currently being used. Please upgrade to add more parallel sessions.
at Server.createSession <node_modules\intern\node_modules\leadfoot\Server.js:307:15>
at <node_modules\intern\lib\executors\Runner.js:174:22>
at retry <node_modules\intern\lib\util.js:695:13>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
Suite chrome 39 on WINDOWS FAILED
UnknownError: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"chrome","version":"39","platform":"WINDOWS","browserstack.local":"true","browserstack.localIdentifier":"1438522680,"browserstack.selenium_version":"2.45.0","name":"tests/intern.js","idle-timeout":60}}] 1 session is currently being used. Please upgrade to add more parallel sessions.
at Server.createSession <node_modules\intern\node_modules\leadfoot\Server.js:307:15>
at <node_modules\intern\lib\executors\Runner.js:174:22>
at retry <node_modules\intern\lib\util.js:695:13>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
Suite chrome 39 on MAC FAILED
UnknownError: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"chrome","version":"39","platform":"MAC","browserstack.local":"true","browserstack.localIdentifier":"1438522680565"owserstack.selenium_version":"2.45.0","name":"tests/intern.js","idle-timeout":60}}] 1 session is currently being used. Please upgrade to add more parallel sessions.
at Server.createSession <node_modules\intern\node_modules\leadfoot\Server.js:307:15>
at <node_modules\intern\lib\executors\Runner.js:174:22>
at retry <node_modules\intern\lib\util.js:695:13>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
Suite safari 8 on MAC FAILED
UnknownError: [POST http://(redacted)@hub.browserstack.com:4444/wd/hub/session / {"desiredCapabilities":{"browserName":"safari","version":"8","platform":"MAC","browserstack.local":"true","browserstack.localIdentifier":"1438522680565",wserstack.selenium_version":"2.45.0","name":"tests/intern.js","idle-timeout":60}}] 1 session is currently being used. Please upgrade to add more parallel sessions.
at Server.createSession <node_modules\intern\node_modules\leadfoot\Server.js:307:15>
at <node_modules\intern\lib\executors\Runner.js:174:22>
at retry <node_modules\intern\lib\util.js:695:13>
at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
at runCallbacks <node_modules\intern\node_modules\dojo\Promise.ts:11:11>
at <node_modules\intern\node_modules\dojo\Promise.ts:317:4>
at run <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
at process._tickCallback <node.js:355:11>
‣ Created session firefox 37 on WINDOWS (ec70159b543956bc4381e76a9827b9a28e18e883)
✓ firefox 37 on WINDOWS - unit tests - hello - greet (0.001s)
Assertion failed: end <= source_len, file src\smalloc.cc, line 280BrowserStack error log:
IE 10 on Windows 8
Firefox 37:
Activity
csnover commentedon Aug 2, 2015
There are multiple issues here which are either transient or due to your machine configuration:
maxConcurrencyto 1. AFAIK open source accounts on BS have 2 parallel VMs—at least ours does—and I think that is the case for the trial accounts too, but if this is not the case then I can update the default concurrency levelThe only actionable thing here on our end I think is to reduce the concurrency level if it turns out trial accounts on BrowserStack have no VM concurrency, so please let me know about that one. Thanks!
dandv commentedon Aug 3, 2015
Thanks Colin. Updating to Node 0.12 fixed the memory allocation error. I've decreased maxConcurrency but that didn't help.
Here's the log with maxConcurrency still 2, and node 0.12. There likely isn't a transient or network problem to Browserstack - I've never managed to get all tests to pass, and this is the 4th time I'm trying. Connectivity is great (albeit not perfect, blame Comcast) otherwise.
And here's the log after setting
maxConcurencyto 1:The test has been stuck here for 10+ minutes, while BrowserStack suggests it has passed:
30 minutes later, I stopped the session in the BrowserStack UI. The Intern test needed a Ctrl+C:
The Chrome errors in BS were
⇒ unknown error: Location must be set before it can be retrieved (Session info: chrome=39.0.2171.65) (Driver info: chromedriver=2.12.301326 (093c7e07b4a916b690e784b0374c7f618f1ea4be),platform=Mac OS X 10.8.5 x86_64) (WARNING: The server did not provide any stacktrace information) Command duration or timeout: 5 milliseconds Build info: version: '2.45.0', revision: '5017cb8', time: '2015-02-26 23:59:50' System info: host: 'mac-66-201-41-33.browserstack.com', ip: '66.201.41.33', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.8.5', java.version: '1.6.0_65' Session ID: f1ad0a2cfffcf9132eab62f3f2180556 Driver info: org.openqa.selenium.chrome.ChromeDriver Capabilities [{platform=MAC, acceptSslCerts=true, javascriptEnabled=true, browserName=chrome, chrome={userDataDir=/var/folders/r0/sjs_h69n7zg2_g_pv4587bt40000gp/T/.org.chromium.Chromium.tEviV4}, rotatable=false, locationContextEnabled=true, mobileEmulationEnabled=false, version=39.0.2171.65, takesHeapSnapshot=true, cssSelectorsEnabled=true, databaseEnabled=false, handlesAlerts=true, browserConnectionEnabled=false, webStorageEnabled=true, nativeEvents=true, applicationCacheEnabled=false, takesScreenshot=true}]Firefox 37 on OS X:
⇒ driver (org.openqa.selenium.firefox.FirefoxDriver) does not support org.openqa.selenium.html5.LocationContext Build info: version: '2.45.0', revision: '5017cb8', time: '2015-02-26 23:59:50' System info: host: 'mac-66-201-41-35.browserstack.com', ip: '66.201.41.35', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.8.5', java.version: '1.6.0_65' Driver info: driver.version: unknownFirefox 37 on XP:
⇒ Underlying driver does not implement advanced user interactions yet.IE on Windows 7:
⇒ org.openqa.selenium.ie.InternetExplorerDriver cannot be cast to org.openqa.selenium.Rotatablecsnover commentedon Aug 3, 2015
Problematically, I am still not able to reproduce the issue you are describing where the
loadedclass does not get added to the page in time for the test to pass. This part of the test passes consistently for me, when running on BrowserStack. You are able to verify that you are not trying to capture/forward requests to ajax.googleapis.com and that you have not modified index.html?The Safari issue is again something I have complained to BrowserStack about in the past and they just passed the buck back to Selenium; the example configuration should not have Safari in the default browser list since SafariDriver is abandoned and basically doesn’t work any more (at least not reliably enough to be used for testing).
The logged errors on the Selenium side are normal, these are the result of feature detection tests.
dandv commentedon Aug 4, 2015
Looks like it's a Windows issue. On a Ubuntu VPS, all tests pass until the test hangs at Safari. Thankfully, BrowserStack has Windows machines to reproduce this issue on ;)
I'm not trying to do anything like that intentionally, but I'm now sure how I might unknowingly do so. I use uBlock but as a Chrome extension, and no system-wide ad blocker of any sort (not that ajax.googleapis.com should be blocked).
I have cloned the repo again and haven't changed anything before running the test. Wish I could paste the complete session verbatim but the "Tunnel download: xx.xxx%" progress takes one line per update (LF instead of CR?) and has eaten up the ConEmu window buffer. But it went like this:
And here's the log from the Ubuntu VPS:
mikehdt commentedon Nov 8, 2015
I've been doing the tutorial as well, but could this also be that I noticed free accounts are restricted to certain browser / OS combinations? I noticed that you couldn't choose most of those combinations of browser / OS. I'm not sure if that's just a restriction of the "live" view versus running through a CLI?