In Selenium Server extension, loops not working

[apodhajsky]

hello,
i have a problem :
i'm trying to run tests writen in SeleniumIDE heavily dependend on SelBlocks (btw briliant work). But when i try to run them they do only 1 loop.

Firefox version: 33.0
Selenium version: 2.9.0
SelBlocks version: 2.1

cmd:
java -jar selenium-server-standalone-2.45.0.jar -port 5555 -firefoxProfileTemplate "full_path_to_template" -singlewindow -debug -log ./htmlsuit.log -htmlSuite "*firefox" "https://address.sk" "/full_path_to_suite/sel_block_test.html" "result.html" -user-extensions "/full_path_to/user-extensions.js"

result:

info: [selblocks] Applying testCase server patch for selblocks
info: Starting test /selenium-server/tests/for_loop_test
info: Executing: |for | i=0; i<=10; i++ | |
info: Executing: |getEval | LOG.info("${i}"); | |
info: script is: LOG.info("0");
info: 0
info: Executing: |endFor | | |
info: [selblocks] branch => @1: [for|i=0; i<=10; i++]
info: Starting test /selenium-server/tests/while_test
info: Executing: |store | 10 | k |
info: Executing: |while | ${k} >= 0 | |
info: Executing: |storeEval | "${k}"-1; | k |
info: script is: "10"-1;
info: Executing: |getEval | LOG.info("${k}"); | |
info: script is: LOG.info("9");
info: 9
info: Executing: |endWhile | | |
info: [selblocks] branch => @2: [while|10 >= 0]

as you can see, after 1st loop test finish

[apodhajsky]

it seems that no command is executed ater endLoop command
selenium standalone server : 2.45.0
Firefox version: 33.0
Selenium version: 2.9.0
SelBlocks version: 2.1

for_loop_test
getEval	LOG.info("------------------------------------------------------------------------------");
store	0	checkVar
for	i=0; i<=10; i++
getEval	LOG.info(${i});
storeEval	(${i});	checkVar
endFor
getEval	LOG.info(${checkVar});
assertEval	${checkVar}	10
getEval	LOG.info("------------------------------------ END ------------------------------------------");

while_test
getEval	LOG.info("------------------------------------ BEGIN ------------------------------------------");
store	10	k
while	${k} >= 0
storeEval	${k}-1;	k
getEval	LOG.info("${k}");
endWhile
getEval	LOG.info("${k}");
assertEval	${k} -1
getEval	LOG.info("------------------------------------ END ------------------------------------------");

result
info: [selblocks] Applying testCase server patch for selblocks
info: Starting test /selenium-server/tests/for_loop_test
info: Executing: |getEval | LOG.info("------------------------------------------------------------------------------"); | |
info: script is: LOG.info("------------------------------------------------------------------------------");
info: ------------------------------------------------------------------------------
info: Executing: |store | 0 | checkVar |
info: Executing: |for | i=0; i<=10; i++ | |
info: Executing: |getEval | LOG.info(${i}); | |
info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP
info: script is: LOG.info(0);
info: 0
info: Executing: |storeEval | (${i}); | checkVar |
info: script is: (0);
info: Executing: |endFor | | |
info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP
info: [selblocks] 2

info: [selblocks] branch => @3: [for|i=0; i<=10; i++]
info: Executing: |for | i=0; i<=10; i++ | |
info: Executing: |getEval | LOG.info(0); | |
info: script is: LOG.info(0);
info: 0
info: Executing: |storeEval | (0); | checkVar |
info: script is: (0);
info: Starting test /selenium-server/tests/while_test
info: Executing: |getEval | LOG.info("------------------------------------ BEGIN ------------------------------------------"); | |
info: script is: LOG.info("------------------------------------ BEGIN ------------------------------------------");
info: ------------------------------------ BEGIN ------------------------------------------
info: Executing: |store | 10 | k |
info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP
info: Executing: |while | ${k} >= 0 | |
info: Executing: |storeEval | ${k}-1; | k |
info: script is: 10-1;
info: Executing: |getEval | LOG.info("${k}"); | |
info: script is: LOG.info("9");
info: 9
info: Executing: |endWhile | | |
info: [selblocks] 2

info: [selblocks] branch => @3: [while|10 >= 0]
info: Executing: |while | 10 >= 0 | |
info: Executing: |storeEval | 10-1; | k |
info: script is: 10-1;
info: Executing: |getEval | LOG.info("9"); | |
info: script is: LOG.info("9");
info: 9

[refactoror]

@apodhajsky
I just don't have time to try to reproduce this.
Please note that the SelBlocks regression test suite covers this scenario, and it works correctly.
Perhaps you could run the test suite in your environment and compare the results.

See: https://github.com/refactoror/SelBlocks/tree/master/sel-blocksTests
The html result files for each browser are also in that directory.

[apodhajsky]

@refactoror
i run regression tests and :
1.) on selenium IDE - all green
2.) on seleniumServer -htmlsuite - all red
after removing log and emmit commands (becouse they are not part of user-extension.js) test dies after 1st cycle (in case of for loops)
again just to be clear, there is error is when i run seleniumServer -htmlsuite in IDE everything works more than well

[refactoror]

SORRY. I forgot to mention that the test suite requires the SelBench plugin, as well. It provides emit, assert, etc. Without those commands, test success is not meaningful.

Get SelBench here: https://github.com/refactoror/SelBench/blob/master/user-extensions.js
Concatenate this to the SelBlocks user-extensions.js, and run again.

By the way, what version of Firefox are you using? The goal is to support the latest ESR version of Firefox, (currently 31) - definitely not the monthly releases.
https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/latest-esr/

One thing I'm concerned about is that Mozilla is planning some major changes to how addons are constrained, some of which may be starting to show up in the monthly releases.
See: https://groups.google.com/forum/#!topic/selenium-users/p24nzz_3p-E
I hope this is not related to that...

[apodhajsky]

sorry for late response i change job recently so less time to spend on this. btw. i think that browser version is irrelevant becouse selenium server is execution engine and not Selenium IDE running in browser. but i have Firefox 37.0.1 and i test it also on 31.6.0
i put selblocks and selbench togeter and yes, report is all green, but report Lies!!!
i cut for.html test from regression testssuite and change cycle to go from 0 to 10 and as you can see at the bottom, it run only ONCE! even emit END is not in log

Test suite results
result: passed
totalTime: 0
numTestTotal: 2
numTestPasses: 2
numTestFailures: 0
numCommandPasses: 0
numCommandFailures: 0
numCommandErrors: 0
Selenium Version: 2.45
Selenium Revision: .0
Test Suite
README
for
README.html
README
clearLog
open http://refactoror.net/_SelBlocks-testpage.html
for.html
for
resetEmitted
deleteVars f
emit "START ${f}"
for s=0,f=0; f <= 10; f++
emit "iter=${f}.${s}"
endFor
emit "END ${f}"
assertEmitted "START $" + "{f}iter=3.0iter=4.0iter=5.0END $" + "{f}"

---

info: [selbench] Applying testCase server patch for selbench
info: Starting test /selenium-server/tests/README.html
info: Executing: |clearLog | | |
info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP
warn: [selbench] clearLog command ignored in non-IDE environment
info: Executing: |open | http://refactoror.net/_SelBlocks-testpage.html | |

---

info: Starting test /selenium-server/tests/for.html
info: Executing: |resetEmitted | | |
info: Executing: |deleteVars | f | |
info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP
info: Executing: |emit | "START ${f}" | |
**************************************** 1st cycle begin ********************************************************
info: Executing: |for | s=0,f=0; f <= 10; f++ | |
info: Executing: |emit | "iter=${f}.${s}" | |
info: Executing: |endFor | | |
**************************************** 1st cycle end ********************************************************
info: [selblocks] branch => @4: [for|s=0,f=0; f <= 10; f++]
**************************************** 2nd cycle begin ********************************************************
info: Executing: |for | s=0,f=0; f <= 10; f++ | |
info: Selenium 1.0 (Core, RC, etc) is no longer under active development. Please update to WebDriver ASAP
info: Executing: |emit | "iter=0.0" | |

[apodhajsky]

as far as i remember by debugging this behaviour it silently dies at command responsibe for execution of html line with for parameters recomputation
and almost same behaviour is when i try emulate loops using if - goto construction
btw. i also compare results and are same: this is from results for firefox from repository:
info: Starting test /selenium-server/tests/for.html
info: Executing: |resetEmitted | | |
info: Executing: |deleteVars | f | |
info: Executing: |emit | "START ${f}" | |
info: Executing: |for | s=0,f=3; f <= 5; f++ | |
info: Executing: |emit | "iter=${f}.${s}" | |
info: Executing: |endFor | | |
info: [selblocks] branch => @4: [for|s=0,f=3; f <= 5; f++]
info: Executing: |for | s=0,f=3; f <= 5; f++ | |
info: Executing: |emit | "iter=3.0" | |
info: Starting test /selenium-server/tests/foreach.html

[refactoror]

Confirmed. Per the original bug description, "no command is executed after endLoop command". The reason that failed tests are reporting as successful is that the assertEmitted is never reached.

This is a huge hole in the testsuite, and every test needs to be re-scrutinized.

@apodhajsky, thanks for your persistence in reporting this.

[apodhajsky]

I need to be persistent, becouse not every tester is also java programmer able to wrile full webdriver test. But almost everybody have some basic programming skills needed to craft solid SeleniumIDE test with SelBlocks. And be able to run html suite on Selenium Grid is final goal that can help lot of people out there...

[medimohammadise]

I have the same problem. After 1'th iteration test case does not run anything.Do you have any update on this bug?
info: [selblocks] branch => @2: [forJson|http://127.0.0.1:7101/INTLFunctionalTestSuite-TestDriverUI-context-root/test2015-05-27_09-08-16.json]

[medimohammadise]

Please Help me

[apodhajsky]

Hi, what you can do right now is to transform your SeleniumIDE (Selblocks) test into JUnit or TestNG test and run them (after few changes) from Eclipse or as jar file...

[refactoror]

Correct. That has been the assumption since the inception of Selblocks. Running directly via the server extension is experimental - more so than I had originally thought. However, we will continue to look in to getting it to work...

[medimohammadise]

I wonder that If I want to convert it to JUnit. How can I use Selblocks's related functionality such as forJson ?

[middreks]

I've bumped into the same issue while trying to run my test suites with Jenkins. Seems to have the same problem both with forXml and loadTestData. Right now we don't have the resources and capacity to transform our tests to JUnit. So could it be worth it to temper with the extension's code to try to make this work?