Hi Lan Liang,
It's true that build Pegasus will consume some memory, you can reduce
the memory consumed with lower parallel by using:
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c -j 1"
In the 2nd step, you can start the cluster by using *** docker run -v
$PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh start_onebox" ***
And, have you tried this way [1] to start a Pegasus cluster?
Your report is very valuable for us, we will improve the docs and make
the demo more easier to use, thanks!
1.
https://github.com/apache/incubator-pegasus/tree/master/docker/pegasus-docker-compose
Best regards,
Yingchun Lai
On Sat, Aug 27, 2022 at 7:43 PM Lan Liang <[email protected]> wrote:
>
> Hi Yingchun,
>
> Thanks for your confirmed.
>
> I have run it success on the github action [0] and recompile pegasus and got
> the different error.
>
> Run command of ** docker run -v $PWD:/root/apache
> apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c"
> **,
>
> And Got error:
>
> The first time,
> ***
> ...
> [ 76%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
>
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o] Error 4
> make[2]: *** Waiting for unfinished jobs....
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
> Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
> [ 80%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make: *** [all] Error 2
> ***
>
>
> The second time,
> ***
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o]
> Error 4
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
> [ 77%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server
> ***
>
> Third time
> ***
> [ 78%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o]
> Error 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
> Error 2
> make: *** [all] Error 2
> ***
>
>
> It's working on Centos7, virtual machine of 6G4U.
>
> Then i upgrade the machine to 10G6U and compile successed !,
>
>
> Finally, ready to run pegasus, and Got nothing...
>
> 1. pack server
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh pack_server"
>
> 2. start service for pegasus
>
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
>
> And return log say:
>
> ***
> ...
> 12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
> 12050K .......... .......... .......... .......... ....... 100% 7.11M=2.9s
>
> 2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz' saved
> [12387614/12387614]
>
> Decompressing zookeeper...
> ZooKeeper JMX enabled by default
> Using config:
> /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
> Starting zookeeper ... STARTED
> Zookeeper started at port 22181
> starting server
> cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server
> config.ini -app_list meta &>result &
> ***
>
> It working for zookeeper and have no any process about pegasus, IMO, It's too
> hard to starte pegasus, this is so frustrating for me :(
>
>
>
>
> [0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Yingchun Lai<[email protected]> |
> | Date | 8/25/2022 23:52 |
> | To | <[email protected]> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Lan Liang,
>
> Thanks to your report! The doc is out of date, we will update it later.
> What is the full command when you got the last error you mentioned
> above. I have tried but not reproduced it by commands:
> ***
> $ git clone [email protected]:apache/incubator-pegasus.git pegasus_test
> $ cd pegasus_test
> $ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c"
> Build start time: Thu Aug 25 15:36:59 UTC 2022
> Start building third-parties...
> ~/apache/thirdparty/build ~/apache/thirdparty
> -- The C compiler identification is GNU 7.3.1
> -- The CXX compiler identification is GNU 7.3.1
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc - skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ - skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Setting up third-parties...
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
> -- USE_JEMALLOC = OFF
> -- Configuring done
> -- Generating done
> CMake Warning:
> Manually-specified variables were not used by the project:
>
> BOOST_ROOT
> Boost_NO_BOOST_CMAKE
> Boost_NO_SYSTEM_PATHS
> ENABLE_GCOV
> ENABLE_GPERF
>
>
> -- Build files have been written to: /root/apache/thirdparty/build
> [ 2%] Creating directories for 'googletest'
> ...
> ***
>
> And is it the full output, it seems the build stage of thirdparty has
> been skipped?
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output
> dsn.layer2.thrift
> ...
>
> Best regards,
> Yingchun Lai
>
> On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <[email protected]> wrote:
>
>
>
> * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or
> directory *.
> The reason is run the source code packaged by windwos in linux, which
> contains the symbol of the windwos file.
>
> Download source again and compile it on the Centos7, also failed. From image
> of apache/pegasus:build-env-centos7.
>
> ***
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output
> dsn.layer2.thrift
> mv output/dsn.layer2_types.h
> /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
> mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
> rm -rf /root/apache/idl/output
>
> done
> Gen git_commit.h ...
> ~/apache/src ~/apache ~/apache
> fatal: Not a git repository (or any parent up to mount point /root/apache)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> PEGASUS_GIT_COMMIT=non-git-repo
> Generating /root/apache/src/include/pegasus/git_commit.h...
> [Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
> ~/apache/src/release__builder ~/apache/src ~/apache ~/apache
> make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command
> not found
> make: *** [cmake_check_build_system] Error 127
> ***
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<[email protected]> |
> | Date | 8/24/2022 10:15 |
> | To | [email protected]<[email protected]> |
> | Subject | Re: How can i run pegasus from container image |
> Just fill the link for [0].
>
> [0] https://pegasus.apache.org/docs/build/compile-by-docker/
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<[email protected]> |
> | Date | 8/23/2022 23:43 |
> | To | [email protected]<[email protected]> |
> | Subject | How can i run pegasus from container image |
> Hi pegasus community:
>
> I want to run pegasus from docker and i follow the doc of compile-by-docker
> [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7
> is not exist.
>
> So i found the image of apache/pegasus:build-env-centos7 and run command of
> *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and
> just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter:
> No such file or directory *.
>
> And it would be great if i can run the pegasus from some docker image.
>
> Remind me if i missed something,Thanks.
>
>
>
>
> Best Regards,
> Lan Liang
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
